[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:02:16 dist-test-slave-lgr5 krb5kdc[12049](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:02:16 dist-test-slave-lgr5 krb5kdc[12049](info): set up 2 sockets
Aug 12 07:02:16 dist-test-slave-lgr5 krb5kdc[12049](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 07:02:17.956606 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:02:18 dist-test-slave-lgr5 krb5kdc[12049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:02:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:02:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:02:18.134053 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:42383
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:34781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:42383
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:18.453399 12068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:18.453685 12068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:18.453815 12068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:18.460820 12068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:02:18.460927 12068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:18.461004 12068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:18.461081 12068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:02:18.461140 12068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:02:18.470463 12068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:34781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:42383
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:42383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:18.471717 12068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:18.472993 12068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:18.484184 12077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:18.484155 12081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:18.484223 12079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:18.485271 12068 server_base.cc:1034] running on GCE node
I20260812 07:02:18.485972 12068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:18.487272 12068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:18.488483 12068 hybrid_clock.cc:648] HybridClock initialized: now 1786518138488463 us; error 77 us; skew 500 ppm
I20260812 07:02:18.488809 12068 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:18 dist-test-slave-lgr5 krb5kdc[12049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518138, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:18.491672 12068 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:02:18.493021 12068 webserver.cc:460] Webserver started at http://127.11.192.254:36305/ using document root <none> and password file <none>
I20260812 07:02:18.493654 12068 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:18.493768 12068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:18.494058 12068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:18.497099 12068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "bb5d5581cac74a52ba7407c66f538774"
format_stamp: "Formatted at 2026-08-12 07:02:18 on dist-test-slave-lgr5"
server_key: "0bd9d17914628673606bca4845d7ad20"
server_key_iv: "fb700d16f4ab323939e46580bed2c189"
server_key_version: "encryptionkey@0"
I20260812 07:02:18.497853 12068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "bb5d5581cac74a52ba7407c66f538774"
format_stamp: "Formatted at 2026-08-12 07:02:18 on dist-test-slave-lgr5"
server_key: "0bd9d17914628673606bca4845d7ad20"
server_key_iv: "fb700d16f4ab323939e46580bed2c189"
server_key_version: "encryptionkey@0"
I20260812 07:02:18.502662 12068 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:02:18.505894 12091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:18.507074 12068 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:02:18.507231 12068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "bb5d5581cac74a52ba7407c66f538774"
format_stamp: "Formatted at 2026-08-12 07:02:18 on dist-test-slave-lgr5"
server_key: "0bd9d17914628673606bca4845d7ad20"
server_key_iv: "fb700d16f4ab323939e46580bed2c189"
server_key_version: "encryptionkey@0"
I20260812 07:02:18.507403 12068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:18.520470 12068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:18.524148 12068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:18.524396 12068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:18.541318 12068 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:42383
I20260812 07:02:18.541340 12183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:42383 every 8 connection(s)
I20260812 07:02:18.542608 12068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:02:18.546144 12184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:18.548730 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 12068
I20260812 07:02:18.549052 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:02:18.551451 12035 external_mini_cluster.cc:1428] Setting key 21f3fb533e48ac594a41e0626ffd870a
I20260812 07:02:18.558725 12184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774: Bootstrap starting.
I20260812 07:02:18.561542 12184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:18.562490 12184 log.cc:824] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:18.565145 12184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774: No bootstrap required, opened a new log
I20260812 07:02:18.574628 12184 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5d5581cac74a52ba7407c66f538774" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 42383 } }
I20260812 07:02:18.575093 12184 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:18.575209 12184 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb5d5581cac74a52ba7407c66f538774, State: Initialized, Role: FOLLOWER
I20260812 07:02:18.575872 12184 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5d5581cac74a52ba7407c66f538774" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 42383 } }
I20260812 07:02:18.576134 12184 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:02:18.576272 12184 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:02:18.576416 12184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:18.578521 12184 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5d5581cac74a52ba7407c66f538774" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 42383 } }
I20260812 07:02:18.578961 12184 leader_election.cc:302] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb5d5581cac74a52ba7407c66f538774; no voters:
I20260812 07:02:18.579531 12184 leader_election.cc:288] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:02:18.579618 12191 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774 [term 1 FOLLOWER]: Leader election won for term 1
Aug 12 07:02:18 dist-test-slave-lgr5 krb5kdc[12049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:18.580875 12191 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774 [term 1 LEADER]: Becoming Leader. State: Replica: bb5d5581cac74a52ba7407c66f538774, State: Running, Role: LEADER
I20260812 07:02:18.581353 12191 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5d5581cac74a52ba7407c66f538774" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 42383 } }
I20260812 07:02:18.581712 12184 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:02:18.584175 12194 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb5d5581cac74a52ba7407c66f538774" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5d5581cac74a52ba7407c66f538774" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 42383 } } }
I20260812 07:02:18.584642 12194 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774 [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:18.584489 12195 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb5d5581cac74a52ba7407c66f538774. Latest consensus state: current_term: 1 leader_uuid: "bb5d5581cac74a52ba7407c66f538774" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5d5581cac74a52ba7407c66f538774" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 42383 } } }
I20260812 07:02:18.585340 12195 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774 [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:18.587770 12203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:02:18.593145 12203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:02:18.595255 12187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:18.556137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52578 (local address 127.11.192.254:42383)
0812 07:02:18.557059 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:18.557074 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:18.558307 (+ 1233us) server_negotiation.cc:407] Connection header received
0812 07:02:18.567434 (+ 9127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:18.567460 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:18.567912 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:18.568350 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:18.570813 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:18.572855 (+ 2042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:18.574405 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:18.575426 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:18.575627 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:18.581690 (+ 6063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:18.581707 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:18.581725 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:18.581766 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:18.587649 (+ 5883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:18.588718 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:18.588727 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:18.588735 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:18.588942 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:18.589534 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:18.589540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:18.589544 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:18.590306 (+ 762us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:18.593081 (+ 2775us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:18.593367 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 07:02:18.593815 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":426,"threads_started":1}
I20260812 07:02:18.606655 12203 catalog_manager.cc:1347] Generated new cluster ID: ca1032eee5c84a70ab061e9d401efacf
I20260812 07:02:18.606765 12203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:02:18.625556 12203 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:02:18.626801 12203 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:02:18.633119 12203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb5d5581cac74a52ba7407c66f538774: Generated new TSK 0
I20260812 07:02:18.633852 12203 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
I20260812 07:02:18.683033 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:42383
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:34781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:19.029131 12223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:19.029394 12223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:19.029533 12223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:19.036504 12223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:19.036610 12223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:19.036764 12223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:02:19.045938 12223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:34781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:42383
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:19.047062 12223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:19.048357 12223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:19.060420 12231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:19.060392 12232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:19.060446 12235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:19.061504 12223 server_base.cc:1034] running on GCE node
I20260812 07:02:19.062027 12223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:19.062829 12223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:19.064013 12223 hybrid_clock.cc:648] HybridClock initialized: now 1786518139063971 us; error 36 us; skew 500 ppm
I20260812 07:02:19.064318 12223 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:19 dist-test-slave-lgr5 krb5kdc[12049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518139, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:19.067170 12223 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:02:19.068570 12223 webserver.cc:460] Webserver started at http://127.11.192.193:35111/ using document root <none> and password file <none>
I20260812 07:02:19.069183 12223 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:19.069299 12223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:19.069602 12223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:19.072363 12223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "df49a140343d446e9f28d891fc4cb226"
format_stamp: "Formatted at 2026-08-12 07:02:19 on dist-test-slave-lgr5"
server_key: "82af478e4f3d760197a67ccc1cb321ce"
server_key_iv: "2f4693356fe4704ea459ccd5c60351b5"
server_key_version: "encryptionkey@0"
I20260812 07:02:19.073077 12223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "df49a140343d446e9f28d891fc4cb226"
format_stamp: "Formatted at 2026-08-12 07:02:19 on dist-test-slave-lgr5"
server_key: "82af478e4f3d760197a67ccc1cb321ce"
server_key_iv: "2f4693356fe4704ea459ccd5c60351b5"
server_key_version: "encryptionkey@0"
I20260812 07:02:19.077819 12223 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 07:02:19.081106 12247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:19.082397 12223 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:02:19.082569 12223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "df49a140343d446e9f28d891fc4cb226"
format_stamp: "Formatted at 2026-08-12 07:02:19 on dist-test-slave-lgr5"
server_key: "82af478e4f3d760197a67ccc1cb321ce"
server_key_iv: "2f4693356fe4704ea459ccd5c60351b5"
server_key_version: "encryptionkey@0"
I20260812 07:02:19.082731 12223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:19.099850 12223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:19.103487 12223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:19.103724 12223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:19.104693 12223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:02:19.106079 12223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:19.106153 12223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:19.106248 12223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:19.106292 12223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:19.137548 12223 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:45681
I20260812 07:02:19.137590 12430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:45681 every 8 connection(s)
I20260812 07:02:19.138875 12223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:02:19.140784 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 12223
I20260812 07:02:19.141008 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:02:19.141691 12035 external_mini_cluster.cc:1428] Setting key a8856da465175c2bbd8c56e636990be4
Aug 12 07:02:19 dist-test-slave-lgr5 krb5kdc[12049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518139, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:19.158784 12434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:19.141439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:33093 (local address 127.11.192.254:42383)
0812 07:02:19.141919 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:19.141926 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:19.143376 (+ 1450us) server_negotiation.cc:407] Connection header received
0812 07:02:19.144989 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:19.144995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:19.145074 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:19.145187 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:19.146602 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:19.147535 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:19.149192 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:19.149735 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:19.149875 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:19.153422 (+ 3547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:19.153427 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:19.153431 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:19.153462 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:19.155882 (+ 2420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:19.156595 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:19.156599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:19.156602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:19.156715 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:19.157125 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:19.157128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:19.157130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:19.157397 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:19.157527 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:19.158205 (+ 678us) server_negotiation.cc:299] Negotiation successful
0812 07:02:19.158434 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":224,"threads_started":1}
WARNING: no policy specified for kudu/127.11.192.194@KRBTEST.COM; defaulting to no policy
I20260812 07:02:19.159644 12436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:19.142035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:42383 (local address 127.11.192.193:33093)
0812 07:02:19.142845 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:19.142892 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:19.144719 (+ 1827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:19.145371 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:19.145378 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:19.145900 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:19.146436 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:19.146454 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:19.147704 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:19.147708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:19.149054 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:19.149068 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:19.150021 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:19.150029 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:19.150315 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:19.150979 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:19.151017 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:19.153211 (+ 2194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:19.156010 (+ 2799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:19.156019 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:19.156033 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:19.156475 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:19.156815 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:19.156820 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:19.156824 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:19.157014 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:19.157518 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:19.157526 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:19.157839 (+ 313us) client_negotiation.cc:769] Sending connection context
0812 07:02:19.158066 (+ 227us) client_negotiation.cc:240] Negotiation successful
0812 07:02:19.158547 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":317,"threads_started":1}
I20260812 07:02:19.161170 12432 heartbeater.cc:347] Connected to a master server at 127.11.192.254:42383
I20260812 07:02:19.161448 12432 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.194@KRBTEST.COM" created.
I20260812 07:02:19.162247 12432 heartbeater.cc:511] Master 127.11.192.254:42383 requested a full tablet report, sending...
I20260812 07:02:19.164328 12119 ts_manager.cc:194] Registered new tserver with Master: df49a140343d446e9f28d891fc4cb226 (127.11.192.193:45681)
I20260812 07:02:19.165756 12119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:33093
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
I20260812 07:02:19.182914 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:42383
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:34781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:19.499222 12443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:19.499480 12443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:19.499612 12443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:19.506721 12443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:19.506829 12443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:19.506994 12443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:02:19.516355 12443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:34781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:42383
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:19.517514 12443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:19.518774 12443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:19.530755 12449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:19.530818 12453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:19.530786 12450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:19.531983 12443 server_base.cc:1034] running on GCE node
I20260812 07:02:19.532558 12443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:19.533447 12443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:19.534643 12443 hybrid_clock.cc:648] HybridClock initialized: now 1786518139534597 us; error 40 us; skew 500 ppm
I20260812 07:02:19.534943 12443 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:19 dist-test-slave-lgr5 krb5kdc[12049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518139, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:19.537880 12443 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:02:19.539183 12443 webserver.cc:460] Webserver started at http://127.11.192.194:36301/ using document root <none> and password file <none>
I20260812 07:02:19.539804 12443 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:19.539930 12443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:19.540262 12443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:19.543069 12443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "5877878994d340d6ba8906f9e51980ee"
format_stamp: "Formatted at 2026-08-12 07:02:19 on dist-test-slave-lgr5"
server_key: "dcc0b41f657ba9946a6d487caec16683"
server_key_iv: "0650f43371a58e326261e78bf0459e8a"
server_key_version: "encryptionkey@0"
I20260812 07:02:19.543862 12443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "5877878994d340d6ba8906f9e51980ee"
format_stamp: "Formatted at 2026-08-12 07:02:19 on dist-test-slave-lgr5"
server_key: "dcc0b41f657ba9946a6d487caec16683"
server_key_iv: "0650f43371a58e326261e78bf0459e8a"
server_key_version: "encryptionkey@0"
I20260812 07:02:19.548750 12443 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:02:19.552134 12462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:19.553380 12443 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 07:02:19.553520 12443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "5877878994d340d6ba8906f9e51980ee"
format_stamp: "Formatted at 2026-08-12 07:02:19 on dist-test-slave-lgr5"
server_key: "dcc0b41f657ba9946a6d487caec16683"
server_key_iv: "0650f43371a58e326261e78bf0459e8a"
server_key_version: "encryptionkey@0"
I20260812 07:02:19.553683 12443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:19.571028 12443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:19.574584 12443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:19.574837 12443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:19.575739 12443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:02:19.577054 12443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:19.577131 12443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:19.577251 12443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:19.577303 12443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:19.607869 12443 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:40821
I20260812 07:02:19.607914 12662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:40821 every 8 connection(s)
I20260812 07:02:19.609375 12443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:02:19.617962 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 12443
I20260812 07:02:19.618135 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:02:19.618556 12035 external_mini_cluster.cc:1428] Setting key f6ea9e354f5183be4047625684eb4ca9
Aug 12 07:02:19 dist-test-slave-lgr5 krb5kdc[12049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518139, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:19.628973 12434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:19.612307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:39931 (local address 127.11.192.254:42383)
0812 07:02:19.612467 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:19.612473 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:19.613119 (+ 646us) server_negotiation.cc:407] Connection header received
0812 07:02:19.614885 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:19.614890 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:19.614974 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:19.615071 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:19.616502 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:19.617224 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:19.618794 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:19.619356 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:19.619488 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:19.623205 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:19.623211 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:19.623216 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:19.623255 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:19.626169 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:19.626828 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:19.626833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:19.626836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:19.626980 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:19.627390 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:19.627393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:19.627395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:19.627653 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:19.627772 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:19.628572 (+ 800us) server_negotiation.cc:299] Negotiation successful
0812 07:02:19.628739 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:02:19.630330 12667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:19.612263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:42383 (local address 127.11.192.194:39931)
0812 07:02:19.612960 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:19.613004 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:19.614626 (+ 1622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:19.615255 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:19.615263 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:19.615750 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:19.616338 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:19.616357 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:19.617333 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:19.617337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:19.618650 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:19.618663 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:19.619564 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:19.619569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:19.619840 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:19.620549 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:19.620576 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:19.622970 (+ 2394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:19.626292 (+ 3322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:19.626299 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:19.626310 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:19.626709 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:19.627077 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:19.627080 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:19.627082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:19.627280 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:19.627766 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:19.627774 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:19.628203 (+ 429us) client_negotiation.cc:769] Sending connection context
0812 07:02:19.628530 (+ 327us) client_negotiation.cc:240] Negotiation successful
0812 07:02:19.628997 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":268,"threads_started":1}
I20260812 07:02:19.631662 12664 heartbeater.cc:347] Connected to a master server at 127.11.192.254:42383
I20260812 07:02:19.631949 12664 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:19.632661 12664 heartbeater.cc:511] Master 127.11.192.254:42383 requested a full tablet report, sending...
I20260812 07:02:19.634027 12119 ts_manager.cc:194] Registered new tserver with Master: 5877878994d340d6ba8906f9e51980ee (127.11.192.194:40821)
I20260812 07:02:19.634804 12119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:39931
WARNING: no policy specified for kudu/127.11.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
I20260812 07:02:19.658900 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:42383
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:34781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:19.971125 12671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:19.971441 12671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:19.971606 12671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:19.980396 12671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:19.980499 12671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:19.980656 12671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:02:19.990559 12671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:34781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:42383
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:19.991724 12671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:19.992995 12671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:20.005120 12682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:20.005249 12683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:20.005228 12689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:20.005959 12671 server_base.cc:1034] running on GCE node
I20260812 07:02:20.006556 12671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:20.007438 12671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:20.008630 12671 hybrid_clock.cc:648] HybridClock initialized: now 1786518140008578 us; error 41 us; skew 500 ppm
I20260812 07:02:20.008922 12671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:20 dist-test-slave-lgr5 krb5kdc[12049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518140, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:20.011894 12671 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:02:20.013392 12671 webserver.cc:460] Webserver started at http://127.11.192.195:45459/ using document root <none> and password file <none>
I20260812 07:02:20.014030 12671 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:20.014150 12671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:20.014437 12671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:20.017232 12671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "c8a8677cbcdf4fe3aadd4a845eb9f0db"
format_stamp: "Formatted at 2026-08-12 07:02:20 on dist-test-slave-lgr5"
server_key: "4663ab0c69d66099570a122ce16c6809"
server_key_iv: "dad57de20b795676365fb6eeb231febc"
server_key_version: "encryptionkey@0"
I20260812 07:02:20.017951 12671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "c8a8677cbcdf4fe3aadd4a845eb9f0db"
format_stamp: "Formatted at 2026-08-12 07:02:20 on dist-test-slave-lgr5"
server_key: "4663ab0c69d66099570a122ce16c6809"
server_key_iv: "dad57de20b795676365fb6eeb231febc"
server_key_version: "encryptionkey@0"
I20260812 07:02:20.022738 12671 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:02:20.025923 12702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:20.027112 12671 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:02:20.027261 12671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "c8a8677cbcdf4fe3aadd4a845eb9f0db"
format_stamp: "Formatted at 2026-08-12 07:02:20 on dist-test-slave-lgr5"
server_key: "4663ab0c69d66099570a122ce16c6809"
server_key_iv: "dad57de20b795676365fb6eeb231febc"
server_key_version: "encryptionkey@0"
I20260812 07:02:20.027436 12671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:20.048425 12671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:20.051977 12671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:20.052246 12671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:20.053125 12671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:02:20.054337 12671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:20.054416 12671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:20.054543 12671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:20.054596 12671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:20.085124 12671 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:43547
I20260812 07:02:20.085171 12873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:43547 every 8 connection(s)
I20260812 07:02:20.086329 12671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:02:20.092530 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 12671
I20260812 07:02:20.092696 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:02:20.093111 12035 external_mini_cluster.cc:1428] Setting key 6c49812643fc4ab37d203806cb464223
Aug 12 07:02:20 dist-test-slave-lgr5 krb5kdc[12049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518140, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:20.104987 12434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:20.088687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:56713 (local address 127.11.192.254:42383)
0812 07:02:20.088858 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:20.088863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:20.090031 (+ 1168us) server_negotiation.cc:407] Connection header received
0812 07:02:20.091838 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:20.091843 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:20.091953 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:20.092053 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:20.093635 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:20.094549 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:20.096468 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:20.096871 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:20.096964 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:20.100005 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:20.100010 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:20.100013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:20.100046 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:20.102563 (+ 2517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:20.103145 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:20.103149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:20.103151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:20.103261 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:20.103591 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:20.103594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:20.103596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:20.103834 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:20.103949 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:20.104572 (+ 623us) server_negotiation.cc:299] Negotiation successful
0812 07:02:20.104728 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:02:20.105855 12877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:20.089176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:42383 (local address 127.11.192.195:56713)
0812 07:02:20.089867 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:20.089913 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:20.091579 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:20.092330 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:20.092337 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:20.092886 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:20.093460 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:20.093493 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:20.094706 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:20.094712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:20.096334 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:20.096347 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:20.097055 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:20.097060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:20.097245 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:20.097798 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:20.097819 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:20.099807 (+ 1988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:20.102659 (+ 2852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:20.102665 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:20.102675 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:20.103065 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:20.103346 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:20.103349 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:20.103351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:20.103518 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:20.103918 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:20.103923 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:20.104243 (+ 320us) client_negotiation.cc:769] Sending connection context
0812 07:02:20.104475 (+ 232us) client_negotiation.cc:240] Negotiation successful
0812 07:02:20.104811 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":262,"threads_started":1}
I20260812 07:02:20.107056 12874 heartbeater.cc:347] Connected to a master server at 127.11.192.254:42383
I20260812 07:02:20.107347 12874 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:20.108040 12874 heartbeater.cc:511] Master 127.11.192.254:42383 requested a full tablet report, sending...
I20260812 07:02:20.109367 12119 ts_manager.cc:194] Registered new tserver with Master: c8a8677cbcdf4fe3aadd4a845eb9f0db (127.11.192.195:43547)
I20260812 07:02:20.110095 12119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:56713
I20260812 07:02:20.120391 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 07:02:20 dist-test-slave-lgr5 krb5kdc[12049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518140, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 07:02:20 dist-test-slave-lgr5 krb5kdc[12049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518140, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.192.193@KRBTEST.COM
I20260812 07:02:20.165736 12885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:20.151272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56170 (local address 127.11.192.193:45681)
0812 07:02:20.151894 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:20.151902 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:20.151932 (+ 30us) server_negotiation.cc:407] Connection header received
0812 07:02:20.152130 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:20.152139 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:20.152341 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:20.152496 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:20.153897 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:20.154890 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:20.156264 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:20.156886 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:20.157007 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:20.160737 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:20.160747 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:20.160760 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:20.160792 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:20.163251 (+ 2459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:20.163860 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:20.163866 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:20.163868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:20.163990 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:20.164381 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:20.164384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:20.164386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:20.164741 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:20.164965 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:20.165233 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 07:02:20.165432 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":298,"threads_started":1}
W20260812 07:02:20.166923 12303 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56170
I20260812 07:02:20.168970 12432 heartbeater.cc:503] Master 127.11.192.254:42383 was elected leader, sending a full tablet report...
Aug 12 07:02:20 dist-test-slave-lgr5 krb5kdc[12049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 07:02:20 dist-test-slave-lgr5 krb5kdc[12049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.193@KRBTEST.COM
I20260812 07:02:20.210451 12885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:20.197732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56174 (local address 127.11.192.193:45681)
0812 07:02:20.197916 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:20.197922 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:20.198248 (+ 326us) server_negotiation.cc:407] Connection header received
0812 07:02:20.198424 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:20.198429 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:20.198499 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:20.198586 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:20.199884 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:20.200622 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:20.201807 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:20.202234 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:20.202343 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:20.206064 (+ 3721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:20.206070 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:20.206074 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:20.206109 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:20.208443 (+ 2334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:20.208982 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:20.208986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:20.208988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:20.209103 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:20.209370 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:20.209373 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:20.209375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:20.209615 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:20.209788 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:20.210000 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 07:02:20.210166 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:02:20.211894 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 12223
I20260812 07:02:20.223064 12885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:20.212874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56190 (local address 127.11.192.193:45681)
0812 07:02:20.213023 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:20.213027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:20.213272 (+ 245us) server_negotiation.cc:407] Connection header received
0812 07:02:20.213404 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:20.213407 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:20.213461 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:20.213551 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:20.214600 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:20.215259 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:20.216416 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:20.216817 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:20.216904 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:20.218874 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:20.218879 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:20.218882 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:20.218910 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:20.221183 (+ 2273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:20.221716 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:20.221720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:20.221722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:20.221835 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:20.222091 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:20.222094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:20.222096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:20.222317 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:20.222496 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:20.222666 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 07:02:20.222811 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:02:20.223729 12423 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:20.305850 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 12223
I20260812 07:02:20.327054 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 12443
Aug 12 07:02:20 dist-test-slave-lgr5 krb5kdc[12049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.194@KRBTEST.COM
I20260812 07:02:20.341658 12899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:20.327887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54288 (local address 127.11.192.194:40821)
0812 07:02:20.328472 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:20.328479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:20.328504 (+ 25us) server_negotiation.cc:407] Connection header received
0812 07:02:20.328576 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:20.328583 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:20.328782 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:20.328908 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:20.329979 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:20.331070 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:20.332331 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:20.332869 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:20.332963 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:20.336711 (+ 3748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:20.336722 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:20.336736 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:20.336772 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:20.339414 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:20.339948 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:20.339954 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:20.339956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:20.340127 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:20.340440 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:20.340443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:20.340445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:20.340765 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:20.341000 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:20.341183 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 07:02:20.341377 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":318,"threads_started":1}
I20260812 07:02:20.342847 12651 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:20.422526 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 12443
I20260812 07:02:20.444166 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 12671
Aug 12 07:02:20 dist-test-slave-lgr5 krb5kdc[12049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.195@KRBTEST.COM
I20260812 07:02:20.459764 12901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:20.445152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58210 (local address 127.11.192.195:43547)
0812 07:02:20.445894 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:20.445904 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:20.445933 (+ 29us) server_negotiation.cc:407] Connection header received
0812 07:02:20.446017 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:20.446026 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:20.446285 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:20.446434 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:20.447699 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:20.448824 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:20.450079 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:20.450650 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:20.450767 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:20.454568 (+ 3801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:20.454579 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:20.454593 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:20.454631 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:20.457191 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:20.457804 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:20.457810 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:20.457813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:20.457937 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:20.458276 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:20.458280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:20.458282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:20.458614 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:20.458893 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:20.459194 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 07:02:20.459448 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":342,"threads_started":1}
I20260812 07:02:20.461073 12868 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:20.541776 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 12671
I20260812 07:02:20.564390 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 12068
I20260812 07:02:20.565511 12178 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:20.666676 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 12068
2026-08-12T07:02:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3949 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:02:20 dist-test-slave-lgr5 krb5kdc[12907](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:02:20 dist-test-slave-lgr5 krb5kdc[12907](info): set up 2 sockets
Aug 12 07:02:20 dist-test-slave-lgr5 krb5kdc[12907](info): commencing operation
krb5kdc: starting...
W20260812 07:02:21.923038 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.000s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:02:22 dist-test-slave-lgr5 krb5kdc[12907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:02:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:02:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:02:22.099632 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:36795
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:42411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:36795
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:22.414551 12926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:22.414832 12926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:22.414964 12926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:22.421969 12926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:02:22.422082 12926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:22.422163 12926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:22.422240 12926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:02:22.422298 12926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:02:22.431885 12926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:42411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:36795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:36795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:22.433194 12926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:22.434440 12926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:22.445605 12941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:22.445660 12937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:22.445675 12934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:22.446412 12926 server_base.cc:1034] running on GCE node
I20260812 07:02:22.447114 12926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:22.448477 12926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:22.449669 12926 hybrid_clock.cc:648] HybridClock initialized: now 1786518142449621 us; error 38 us; skew 500 ppm
I20260812 07:02:22.449972 12926 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:22 dist-test-slave-lgr5 krb5kdc[12907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518142, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:22.452903 12926 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:02:22.454360 12926 webserver.cc:460] Webserver started at http://127.11.192.254:43197/ using document root <none> and password file <none>
I20260812 07:02:22.455204 12926 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:22.455349 12926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:22.455705 12926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:22.458670 12926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "d6b6651475d24326b92f938521eb4b91"
format_stamp: "Formatted at 2026-08-12 07:02:22 on dist-test-slave-lgr5"
server_key: "1d2726253f1b72925e582ecdc9a67f06"
server_key_iv: "f1082db57cab3c0c954242815e89d746"
server_key_version: "encryptionkey@0"
I20260812 07:02:22.459401 12926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "d6b6651475d24326b92f938521eb4b91"
format_stamp: "Formatted at 2026-08-12 07:02:22 on dist-test-slave-lgr5"
server_key: "1d2726253f1b72925e582ecdc9a67f06"
server_key_iv: "f1082db57cab3c0c954242815e89d746"
server_key_version: "encryptionkey@0"
I20260812 07:02:22.464965 12926 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260812 07:02:22.468470 12951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:22.469717 12926 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 07:02:22.469880 12926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "d6b6651475d24326b92f938521eb4b91"
format_stamp: "Formatted at 2026-08-12 07:02:22 on dist-test-slave-lgr5"
server_key: "1d2726253f1b72925e582ecdc9a67f06"
server_key_iv: "f1082db57cab3c0c954242815e89d746"
server_key_version: "encryptionkey@0"
I20260812 07:02:22.470084 12926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:22.482787 12926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:22.486603 12926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:22.486867 12926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:22.503872 12926 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:36795
I20260812 07:02:22.503894 13031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:36795 every 8 connection(s)
I20260812 07:02:22.505254 12926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:02:22.508811 13033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:22.513463 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 12926
I20260812 07:02:22.513701 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:02:22.514530 12035 external_mini_cluster.cc:1428] Setting key 370d0c0f153158b8747204e7e38c552c
I20260812 07:02:22.520267 13033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91: Bootstrap starting.
I20260812 07:02:22.523859 13033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:22.525105 13033 log.cc:824] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:22.527698 13033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91: No bootstrap required, opened a new log
Aug 12 07:02:22 dist-test-slave-lgr5 krb5kdc[12907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:22.537627 13037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:22.516589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55912 (local address 127.11.192.254:36795)
0812 07:02:22.517404 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:22.517415 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:22.517459 (+ 44us) server_negotiation.cc:407] Connection header received
0812 07:02:22.519168 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:22.519192 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:22.519537 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:22.519871 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:22.521659 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:22.523152 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:22.524887 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:22.525625 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:22.525822 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:22.529509 (+ 3687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:22.529520 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:22.529538 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:22.529586 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:22.533388 (+ 3802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:22.533991 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:22.533999 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:22.534007 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:22.534221 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:22.534535 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:22.534540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:22.534543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:22.535247 (+ 704us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:22.535586 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:22.535907 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 07:02:22.536360 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":370,"threads_started":1}
I20260812 07:02:22.537561 13033 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6b6651475d24326b92f938521eb4b91" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36795 } }
I20260812 07:02:22.538018 13033 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:22.538134 13033 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6b6651475d24326b92f938521eb4b91, State: Initialized, Role: FOLLOWER
I20260812 07:02:22.538728 13033 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6b6651475d24326b92f938521eb4b91" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36795 } }
I20260812 07:02:22.539033 13033 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:02:22.539186 13033 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:02:22.539343 13033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:22.541733 13033 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6b6651475d24326b92f938521eb4b91" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36795 } }
I20260812 07:02:22.542184 13033 leader_election.cc:302] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6b6651475d24326b92f938521eb4b91; no voters:
I20260812 07:02:22.542719 13033 leader_election.cc:288] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:02:22.542845 13039 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:02:22.544029 13039 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91 [term 1 LEADER]: Becoming Leader. State: Replica: d6b6651475d24326b92f938521eb4b91, State: Running, Role: LEADER
I20260812 07:02:22.544509 13039 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6b6651475d24326b92f938521eb4b91" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36795 } }
I20260812 07:02:22.544780 13033 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:02:22.547212 13041 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6b6651475d24326b92f938521eb4b91. Latest consensus state: current_term: 1 leader_uuid: "d6b6651475d24326b92f938521eb4b91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6b6651475d24326b92f938521eb4b91" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36795 } } }
I20260812 07:02:22.547250 13040 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6b6651475d24326b92f938521eb4b91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6b6651475d24326b92f938521eb4b91" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36795 } } }
I20260812 07:02:22.547580 13041 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91 [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:22.547588 13040 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91 [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:22.549580 13058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:02:22.554852 13058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:02:22.564867 13058 catalog_manager.cc:1347] Generated new cluster ID: f580ddcd27f0415981f0a4e96e316c38
I20260812 07:02:22.564965 13058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:02:22.584240 13058 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:02:22.586036 13058 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:02:22.597545 13058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6b6651475d24326b92f938521eb4b91: Generated new TSK 0
I20260812 07:02:22.598481 13058 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
I20260812 07:02:22.643457 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36795
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:42411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:22.975211 13082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:22.975469 13082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:22.975601 13082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:22.982604 13082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:22.982704 13082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:22.982865 13082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:02:22.992430 13082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:42411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36795
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:22.993579 13082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:22.994804 13082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:23.007097 13094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:23.007157 13098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:23.007144 13095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:23.008036 13082 server_base.cc:1034] running on GCE node
I20260812 07:02:23.008626 13082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:23.009516 13082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:23.010705 13082 hybrid_clock.cc:648] HybridClock initialized: now 1786518143010665 us; error 34 us; skew 500 ppm
I20260812 07:02:23.010991 13082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:23 dist-test-slave-lgr5 krb5kdc[12907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518143, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:23.014014 13082 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:02:23.015373 13082 webserver.cc:460] Webserver started at http://127.11.192.193:35663/ using document root <none> and password file <none>
I20260812 07:02:23.016009 13082 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:23.016146 13082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:23.016461 13082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:23.019255 13082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "f5b3c13e5fc549269a7da6a7a081160e"
format_stamp: "Formatted at 2026-08-12 07:02:23 on dist-test-slave-lgr5"
server_key: "11ede88de29a5ef689ef5238e9cac759"
server_key_iv: "b6aad537e5fa9871e8d2c8cf10d246aa"
server_key_version: "encryptionkey@0"
I20260812 07:02:23.020033 13082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "f5b3c13e5fc549269a7da6a7a081160e"
format_stamp: "Formatted at 2026-08-12 07:02:23 on dist-test-slave-lgr5"
server_key: "11ede88de29a5ef689ef5238e9cac759"
server_key_iv: "b6aad537e5fa9871e8d2c8cf10d246aa"
server_key_version: "encryptionkey@0"
I20260812 07:02:23.024878 13082 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 07:02:23.028174 13107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:23.029397 13082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:02:23.029546 13082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "f5b3c13e5fc549269a7da6a7a081160e"
format_stamp: "Formatted at 2026-08-12 07:02:23 on dist-test-slave-lgr5"
server_key: "11ede88de29a5ef689ef5238e9cac759"
server_key_iv: "b6aad537e5fa9871e8d2c8cf10d246aa"
server_key_version: "encryptionkey@0"
I20260812 07:02:23.029722 13082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:23.055068 13082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:23.058660 13082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:23.058907 13082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:23.059798 13082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:02:23.061087 13082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:23.061164 13082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:23.061280 13082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:23.061340 13082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:23.090492 13082 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:43921
I20260812 07:02:23.090575 13310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:43921 every 8 connection(s)
I20260812 07:02:23.091893 13082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:02:23.092833 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 13082
I20260812 07:02:23.093062 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:02:23.093806 12035 external_mini_cluster.cc:1428] Setting key 3bc7c2a7c8b074dca3c57812c3e0ed73
Aug 12 07:02:23 dist-test-slave-lgr5 krb5kdc[12907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518143, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:23.114753 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:23.095035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:35893 (local address 127.11.192.254:36795)
0812 07:02:23.100367 (+ 5332us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:23.100374 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:23.100394 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:02:23.100453 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:23.100457 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:23.100541 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:23.100652 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:23.102811 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:23.103735 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:23.105413 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:23.105830 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:23.105932 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:23.109130 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:23.109135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:23.109138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:23.109169 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:23.111615 (+ 2446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:23.112368 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:23.112374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:23.112378 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:23.112537 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:23.112963 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:23.112974 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:23.112978 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:23.113312 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:23.113461 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:23.114137 (+ 676us) server_negotiation.cc:299] Negotiation successful
0812 07:02:23.114366 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5201,"thread_start_us":9056,"threads_started":1}
WARNING: no policy specified for kudu/127.11.192.194@KRBTEST.COM; defaulting to no policy
I20260812 07:02:23.115489 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:23.095369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:36795 (local address 127.11.192.193:35893)
0812 07:02:23.096241 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:23.096287 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:23.098314 (+ 2027us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:23.100994 (+ 2680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:23.101008 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:23.101891 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:23.102629 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:23.102649 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:23.103928 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:23.103933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:23.105252 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:23.105272 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:23.106011 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:23.106016 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:23.106207 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:23.106816 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:23.106839 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:23.108919 (+ 2080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:23.111747 (+ 2828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:23.111754 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:23.111765 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:23.112239 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:23.112654 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:23.112657 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:23.112660 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:23.112850 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:23.113421 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:23.113426 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:23.113720 (+ 294us) client_negotiation.cc:769] Sending connection context
0812 07:02:23.113963 (+ 243us) client_negotiation.cc:240] Negotiation successful
0812 07:02:23.114339 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":354,"threads_started":1}
I20260812 07:02:23.117062 13312 heartbeater.cc:347] Connected to a master server at 127.11.192.254:36795
I20260812 07:02:23.117362 13312 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.194@KRBTEST.COM" created.
I20260812 07:02:23.118136 13312 heartbeater.cc:511] Master 127.11.192.254:36795 requested a full tablet report, sending...
I20260812 07:02:23.120162 12977 ts_manager.cc:194] Registered new tserver with Master: f5b3c13e5fc549269a7da6a7a081160e (127.11.192.193:43921)
I20260812 07:02:23.121585 12977 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:35893
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
I20260812 07:02:23.138796 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36795
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:42411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:23.449888 13325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:23.450119 13325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:23.450217 13325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:23.457150 13325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:23.457238 13325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:23.457358 13325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:02:23.466212 13325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:42411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36795
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:23.467257 13325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:23.468631 13325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:23.480767 13333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:23.480887 13332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:23.480854 13335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:23.481611 13325 server_base.cc:1034] running on GCE node
I20260812 07:02:23.482179 13325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:23.483024 13325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:23.484165 13325 hybrid_clock.cc:648] HybridClock initialized: now 1786518143484113 us; error 45 us; skew 500 ppm
I20260812 07:02:23.484582 13325 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:23 dist-test-slave-lgr5 krb5kdc[12907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518143, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:23.487708 13325 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:02:23.489148 13325 webserver.cc:460] Webserver started at http://127.11.192.194:41455/ using document root <none> and password file <none>
I20260812 07:02:23.489784 13325 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:23.489905 13325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:23.490188 13325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:23.493178 13325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "02f1fee168de40df9bf12e9424812f5a"
format_stamp: "Formatted at 2026-08-12 07:02:23 on dist-test-slave-lgr5"
server_key: "99174ef0fbd8260f75dba0c739409b03"
server_key_iv: "64c563021e97f015fda8f881a2a202de"
server_key_version: "encryptionkey@0"
I20260812 07:02:23.493973 13325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "02f1fee168de40df9bf12e9424812f5a"
format_stamp: "Formatted at 2026-08-12 07:02:23 on dist-test-slave-lgr5"
server_key: "99174ef0fbd8260f75dba0c739409b03"
server_key_iv: "64c563021e97f015fda8f881a2a202de"
server_key_version: "encryptionkey@0"
I20260812 07:02:23.499150 13325 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 07:02:23.502693 13347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:23.504256 13325 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 07:02:23.504419 13325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "02f1fee168de40df9bf12e9424812f5a"
format_stamp: "Formatted at 2026-08-12 07:02:23 on dist-test-slave-lgr5"
server_key: "99174ef0fbd8260f75dba0c739409b03"
server_key_iv: "64c563021e97f015fda8f881a2a202de"
server_key_version: "encryptionkey@0"
I20260812 07:02:23.504637 13325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:23.531353 13325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:23.535643 13325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:23.535925 13325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:23.537011 13325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:02:23.538367 13325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:23.538455 13325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:23.538556 13325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:23.538605 13325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:23.570050 13325 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:39449
I20260812 07:02:23.570089 13537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:39449 every 8 connection(s)
I20260812 07:02:23.571374 13325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:02:23.573282 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 13325
I20260812 07:02:23.573480 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:02:23.574003 12035 external_mini_cluster.cc:1428] Setting key b33d64dad1f20c255ff18aed136ab129
Aug 12 07:02:23 dist-test-slave-lgr5 krb5kdc[12907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518143, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.192.195@KRBTEST.COM; defaulting to no policy
I20260812 07:02:23.592515 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:23.574153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:49781 (local address 127.11.192.254:36795)
0812 07:02:23.574363 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:23.574370 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:23.576322 (+ 1952us) server_negotiation.cc:407] Connection header received
0812 07:02:23.578281 (+ 1959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:23.578286 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:23.578362 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:23.578469 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:23.579978 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:23.580902 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:23.582475 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:23.582959 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:23.583081 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:23.586600 (+ 3519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:23.586606 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:23.586610 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:23.586647 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:23.589321 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:23.590054 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:23.590060 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:23.590062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:23.590202 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:23.590651 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:23.590656 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:23.590658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:23.590938 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:23.591083 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:23.591829 (+ 746us) server_negotiation.cc:299] Negotiation successful
0812 07:02:23.592070 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:02:23.593420 13542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:23.574592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:36795 (local address 127.11.192.194:49781)
0812 07:02:23.576146 (+ 1554us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:23.576206 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:23.578012 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:23.578676 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:23.578685 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:23.579200 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:23.579799 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:23.579818 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:23.581037 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:23.581041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:23.582315 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:23.582329 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:23.583205 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:23.583213 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:23.583457 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:23.584170 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:23.584196 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:23.586338 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:23.589469 (+ 3131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:23.589479 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:23.589495 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:23.589931 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:23.590326 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:23.590332 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:23.590336 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:23.590534 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:23.591072 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:23.591089 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:23.591420 (+ 331us) client_negotiation.cc:769] Sending connection context
0812 07:02:23.591696 (+ 276us) client_negotiation.cc:240] Negotiation successful
0812 07:02:23.592229 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1238,"thread_start_us":997,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.195@KRBTEST.COM" created.
I20260812 07:02:23.594869 13538 heartbeater.cc:347] Connected to a master server at 127.11.192.254:36795
I20260812 07:02:23.595149 13538 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:23.595819 13538 heartbeater.cc:511] Master 127.11.192.254:36795 requested a full tablet report, sending...
I20260812 07:02:23.597307 12977 ts_manager.cc:194] Registered new tserver with Master: 02f1fee168de40df9bf12e9424812f5a (127.11.192.194:39449)
I20260812 07:02:23.598233 12977 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:49781
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
I20260812 07:02:23.615084 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36795
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:42411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:23.927889 13547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:23.928154 13547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:23.928288 13547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:23.935307 13547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:23.935416 13547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:23.935573 13547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:02:23.944963 13547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:42411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36795
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:23.946287 13547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:23.947651 13547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:23.959919 13563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:23.959961 13559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:23.959880 13561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:23.960975 13547 server_base.cc:1034] running on GCE node
I20260812 07:02:23.961568 13547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:23.962476 13547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:23.963681 13547 hybrid_clock.cc:648] HybridClock initialized: now 1786518143963625 us; error 46 us; skew 500 ppm
I20260812 07:02:23.963964 13547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:23 dist-test-slave-lgr5 krb5kdc[12907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518143, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:23.966822 13547 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:02:23.968185 13547 webserver.cc:460] Webserver started at http://127.11.192.195:38179/ using document root <none> and password file <none>
I20260812 07:02:23.968801 13547 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:23.968917 13547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:23.969228 13547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:23.972628 13547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "c0e7b5e9592f4065bedee61976a1e02f"
format_stamp: "Formatted at 2026-08-12 07:02:23 on dist-test-slave-lgr5"
server_key: "2604b0b875645ed85d9f8a7465b244c9"
server_key_iv: "5e9fbc295caa5ee7b6f25af6a624979c"
server_key_version: "encryptionkey@0"
I20260812 07:02:23.973591 13547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "c0e7b5e9592f4065bedee61976a1e02f"
format_stamp: "Formatted at 2026-08-12 07:02:23 on dist-test-slave-lgr5"
server_key: "2604b0b875645ed85d9f8a7465b244c9"
server_key_iv: "5e9fbc295caa5ee7b6f25af6a624979c"
server_key_version: "encryptionkey@0"
I20260812 07:02:23.979696 13547 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260812 07:02:23.982995 13573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:23.984244 13547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 07:02:23.984385 13547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "c0e7b5e9592f4065bedee61976a1e02f"
format_stamp: "Formatted at 2026-08-12 07:02:23 on dist-test-slave-lgr5"
server_key: "2604b0b875645ed85d9f8a7465b244c9"
server_key_iv: "5e9fbc295caa5ee7b6f25af6a624979c"
server_key_version: "encryptionkey@0"
I20260812 07:02:23.984565 13547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:24.004459 13547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:24.008507 13547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:24.008759 13547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:24.009703 13547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:02:24.011085 13547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:24.011154 13547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:24.011267 13547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:24.011314 13547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:24.038842 13547 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:45421
I20260812 07:02:24.038882 13764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:45421 every 8 connection(s)
I20260812 07:02:24.040066 13547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:02:24.040508 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 13547
I20260812 07:02:24.040726 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:02:24.041287 12035 external_mini_cluster.cc:1428] Setting key 0c2e9a925f4e74f277b5a05e4f986ee3
Aug 12 07:02:24 dist-test-slave-lgr5 krb5kdc[12907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518143, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:24.059713 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:24.042671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:54041 (local address 127.11.192.254:36795)
0812 07:02:24.042869 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:24.042875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:24.044274 (+ 1399us) server_negotiation.cc:407] Connection header received
0812 07:02:24.046224 (+ 1950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:24.046228 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:24.046292 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:24.046372 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:24.047986 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.048790 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.050515 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.050902 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.050994 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:24.054210 (+ 3216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:24.054215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:24.054219 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:24.054251 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:24.057284 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:24.057928 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:24.057931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:24.057934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:24.058055 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:24.058375 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:24.058378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:24.058380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:24.058632 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:24.058739 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:24.059288 (+ 549us) server_negotiation.cc:299] Negotiation successful
0812 07:02:24.059440 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:02:24.060631 13768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:24.043291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:36795 (local address 127.11.192.195:54041)
0812 07:02:24.043993 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:24.044038 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:24.045988 (+ 1950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:24.046664 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:24.046673 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:24.047198 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:24.047840 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:24.047861 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.048941 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.048945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:24.050377 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:24.050392 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.051089 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.051094 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:24.051326 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:24.051949 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:24.051975 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:24.054039 (+ 2064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:24.057414 (+ 3375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:24.057421 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:24.057431 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:24.057839 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:24.058129 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:24.058132 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:24.058134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:24.058300 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:24.058719 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:24.058724 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:24.059019 (+ 295us) client_negotiation.cc:769] Sending connection context
0812 07:02:24.059238 (+ 219us) client_negotiation.cc:240] Negotiation successful
0812 07:02:24.059587 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":307,"threads_started":1}
I20260812 07:02:24.061777 13765 heartbeater.cc:347] Connected to a master server at 127.11.192.254:36795
I20260812 07:02:24.062047 13765 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:24.062728 13765 heartbeater.cc:511] Master 127.11.192.254:36795 requested a full tablet report, sending...
I20260812 07:02:24.064013 12977 ts_manager.cc:194] Registered new tserver with Master: c0e7b5e9592f4065bedee61976a1e02f (127.11.192.195:45421)
I20260812 07:02:24.064803 12977 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:54041
I20260812 07:02:24.068781 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:02:24.086419 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:24.074895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55922 (local address 127.11.192.254:36795)
0812 07:02:24.075038 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:24.075051 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:24.075524 (+ 473us) server_negotiation.cc:407] Connection header received
0812 07:02:24.075674 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:24.075679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:24.075738 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:24.075815 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:24.077272 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.077956 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.079207 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.079602 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.079713 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:24.081853 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:24.081858 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:24.081861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:24.081890 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:24.083967 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:24.084601 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:24.084606 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:24.084610 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:24.084733 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:24.085080 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:24.085084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:24.085086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:24.085375 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:24.085597 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:24.085926 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 07:02:24.086139 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 07:02:24.092666 12977 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:02:24.099351 12977 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:02:24.124245 13312 heartbeater.cc:503] Master 127.11.192.254:36795 was elected leader, sending a full tablet report...
I20260812 07:02:24.129925 13790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:24.111307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49014 (local address 127.11.192.194:39449)
0812 07:02:24.112131 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:24.112141 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:24.121880 (+ 9739us) server_negotiation.cc:407] Connection header received
0812 07:02:24.122063 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:24.122072 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:24.122368 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:24.122572 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:24.123665 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.125167 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.127521 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.128665 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.128868 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:24.129073 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:24.129357 (+ 284us) server_negotiation.cc:299] Negotiation successful
0812 07:02:24.129536 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":360,"threads_started":1}
I20260812 07:02:24.130312 13788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:24.110729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:39449 (local address 127.0.0.1:49014)
0812 07:02:24.121732 (+ 11003us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:24.121753 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:24.121900 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:24.123060 (+ 1160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:24.123069 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:24.123101 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:24.123464 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:24.123476 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.125289 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.125295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:24.127338 (+ 2043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:24.127356 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.128919 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.128926 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:24.129122 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:24.129168 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 07:02:24.129290 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 07:02:24.129489 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10870,"mutex_wait_us":23,"thread_start_us":11270,"threads_started":1}
I20260812 07:02:24.132064 13792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:24.112742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35662 (local address 127.11.192.195:45421)
0812 07:02:24.122777 (+ 10035us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:24.122786 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:24.123287 (+ 501us) server_negotiation.cc:407] Connection header received
0812 07:02:24.123399 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:24.123406 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:24.123649 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:24.123840 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:24.126045 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.127524 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.129986 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.131096 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.131278 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:24.131438 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:24.131607 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 07:02:24.131759 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9853,"thread_start_us":9999,"threads_started":1}
I20260812 07:02:24.132112 13791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:24.112352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:45421 (local address 127.0.0.1:35662)
0812 07:02:24.123001 (+ 10649us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:24.123019 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:24.123137 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:24.124269 (+ 1132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:24.124274 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:24.124304 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:24.124812 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:24.124826 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.127671 (+ 2845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.127677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:24.129794 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:24.129807 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.131283 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.131289 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:24.131447 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:24.131477 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 07:02:24.131552 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 07:02:24.131675 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10512,"mutex_wait_us":20,"thread_start_us":10872,"threads_started":1}
I20260812 07:02:24.132573 13786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:24.110174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:43921 (local address 127.0.0.1:47598)
0812 07:02:24.110879 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:24.110919 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:24.111110 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:24.123832 (+ 12722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:24.123836 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:24.123859 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:24.124463 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:24.124483 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.126481 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.126487 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:24.128816 (+ 2329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:24.128835 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.131737 (+ 2902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.131745 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:24.131946 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:24.131979 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:02:24.132052 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 07:02:24.132207 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":62,"thread_start_us":317,"threads_started":1}
I20260812 07:02:24.133543 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:24.110388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47598 (local address 127.11.192.193:43921)
0812 07:02:24.123376 (+ 12988us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:24.123384 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:24.123410 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:02:24.123484 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:24.123490 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:24.123656 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:24.123817 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:24.124731 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.126288 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.129019 (+ 2731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.131481 (+ 2462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.132877 (+ 1396us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:24.133035 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:24.133099 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 07:02:24.133243 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12817,"thread_start_us":12670,"threads_started":1}
I20260812 07:02:24.134845 13419 tablet_service.cc:1467] Processing CreateTablet for tablet b17868b8988e40f5ab89f72bcf5f7625 (DEFAULT_TABLE table=test-table [id=62046020b87445a8a781f202dcd9c286]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:24.136147 13419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b17868b8988e40f5ab89f72bcf5f7625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:24.136126 13655 tablet_service.cc:1467] Processing CreateTablet for tablet b17868b8988e40f5ab89f72bcf5f7625 (DEFAULT_TABLE table=test-table [id=62046020b87445a8a781f202dcd9c286]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:24.137573 13655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b17868b8988e40f5ab89f72bcf5f7625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:24.137243 13190 tablet_service.cc:1467] Processing CreateTablet for tablet b17868b8988e40f5ab89f72bcf5f7625 (DEFAULT_TABLE table=test-table [id=62046020b87445a8a781f202dcd9c286]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:24.139637 13190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b17868b8988e40f5ab89f72bcf5f7625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:24.147840 13793 tablet_bootstrap.cc:492] T b17868b8988e40f5ab89f72bcf5f7625 P 02f1fee168de40df9bf12e9424812f5a: Bootstrap starting.
I20260812 07:02:24.151234 13795 tablet_bootstrap.cc:492] T b17868b8988e40f5ab89f72bcf5f7625 P c0e7b5e9592f4065bedee61976a1e02f: Bootstrap starting.
I20260812 07:02:24.152449 13793 tablet_bootstrap.cc:654] T b17868b8988e40f5ab89f72bcf5f7625 P 02f1fee168de40df9bf12e9424812f5a: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:24.153611 13793 log.cc:824] T b17868b8988e40f5ab89f72bcf5f7625 P 02f1fee168de40df9bf12e9424812f5a: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:24.154866 13795 tablet_bootstrap.cc:654] T b17868b8988e40f5ab89f72bcf5f7625 P c0e7b5e9592f4065bedee61976a1e02f: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:24.155088 13798 tablet_bootstrap.cc:492] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e: Bootstrap starting.
I20260812 07:02:24.156121 13795 log.cc:824] T b17868b8988e40f5ab89f72bcf5f7625 P c0e7b5e9592f4065bedee61976a1e02f: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:24.157909 13793 tablet_bootstrap.cc:492] T b17868b8988e40f5ab89f72bcf5f7625 P 02f1fee168de40df9bf12e9424812f5a: No bootstrap required, opened a new log
I20260812 07:02:24.158213 13793 ts_tablet_manager.cc:1397] T b17868b8988e40f5ab89f72bcf5f7625 P 02f1fee168de40df9bf12e9424812f5a: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260812 07:02:24.158298 13798 tablet_bootstrap.cc:654] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:24.158576 13795 tablet_bootstrap.cc:492] T b17868b8988e40f5ab89f72bcf5f7625 P c0e7b5e9592f4065bedee61976a1e02f: No bootstrap required, opened a new log
I20260812 07:02:24.158799 13795 ts_tablet_manager.cc:1397] T b17868b8988e40f5ab89f72bcf5f7625 P c0e7b5e9592f4065bedee61976a1e02f: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260812 07:02:24.159487 13798 log.cc:824] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:24.165786 13795 raft_consensus.cc:348] T b17868b8988e40f5ab89f72bcf5f7625 P c0e7b5e9592f4065bedee61976a1e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b3c13e5fc549269a7da6a7a081160e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43921 } } peers { permanent_uuid: "c0e7b5e9592f4065bedee61976a1e02f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 45421 } } peers { permanent_uuid: "02f1fee168de40df9bf12e9424812f5a" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 39449 } }
I20260812 07:02:24.166122 13795 raft_consensus.cc:374] T b17868b8988e40f5ab89f72bcf5f7625 P c0e7b5e9592f4065bedee61976a1e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:24.166213 13795 raft_consensus.cc:729] T b17868b8988e40f5ab89f72bcf5f7625 P c0e7b5e9592f4065bedee61976a1e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0e7b5e9592f4065bedee61976a1e02f, State: Initialized, Role: FOLLOWER
I20260812 07:02:24.166975 13795 consensus_queue.cc:260] T b17868b8988e40f5ab89f72bcf5f7625 P c0e7b5e9592f4065bedee61976a1e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b3c13e5fc549269a7da6a7a081160e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43921 } } peers { permanent_uuid: "c0e7b5e9592f4065bedee61976a1e02f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 45421 } } peers { permanent_uuid: "02f1fee168de40df9bf12e9424812f5a" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 39449 } }
I20260812 07:02:24.168059 13765 heartbeater.cc:503] Master 127.11.192.254:36795 was elected leader, sending a full tablet report...
I20260812 07:02:24.168725 13795 ts_tablet_manager.cc:1428] T b17868b8988e40f5ab89f72bcf5f7625 P c0e7b5e9592f4065bedee61976a1e02f: Time spent starting tablet: real 0.010s user 0.001s sys 0.009s
I20260812 07:02:24.168968 13798 tablet_bootstrap.cc:492] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e: No bootstrap required, opened a new log
I20260812 07:02:24.169265 13798 ts_tablet_manager.cc:1397] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.007s
I20260812 07:02:24.168993 13793 raft_consensus.cc:348] T b17868b8988e40f5ab89f72bcf5f7625 P 02f1fee168de40df9bf12e9424812f5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b3c13e5fc549269a7da6a7a081160e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43921 } } peers { permanent_uuid: "c0e7b5e9592f4065bedee61976a1e02f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 45421 } } peers { permanent_uuid: "02f1fee168de40df9bf12e9424812f5a" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 39449 } }
I20260812 07:02:24.169445 13793 raft_consensus.cc:374] T b17868b8988e40f5ab89f72bcf5f7625 P 02f1fee168de40df9bf12e9424812f5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:24.169544 13793 raft_consensus.cc:729] T b17868b8988e40f5ab89f72bcf5f7625 P 02f1fee168de40df9bf12e9424812f5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02f1fee168de40df9bf12e9424812f5a, State: Initialized, Role: FOLLOWER
I20260812 07:02:24.170178 13793 consensus_queue.cc:260] T b17868b8988e40f5ab89f72bcf5f7625 P 02f1fee168de40df9bf12e9424812f5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b3c13e5fc549269a7da6a7a081160e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43921 } } peers { permanent_uuid: "c0e7b5e9592f4065bedee61976a1e02f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 45421 } } peers { permanent_uuid: "02f1fee168de40df9bf12e9424812f5a" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 39449 } }
I20260812 07:02:24.171269 13538 heartbeater.cc:503] Master 127.11.192.254:36795 was elected leader, sending a full tablet report...
I20260812 07:02:24.171890 13793 ts_tablet_manager.cc:1428] T b17868b8988e40f5ab89f72bcf5f7625 P 02f1fee168de40df9bf12e9424812f5a: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260812 07:02:24.178362 13798 raft_consensus.cc:348] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b3c13e5fc549269a7da6a7a081160e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43921 } } peers { permanent_uuid: "c0e7b5e9592f4065bedee61976a1e02f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 45421 } } peers { permanent_uuid: "02f1fee168de40df9bf12e9424812f5a" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 39449 } }
I20260812 07:02:24.178680 13798 raft_consensus.cc:374] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:24.178750 13798 raft_consensus.cc:729] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5b3c13e5fc549269a7da6a7a081160e, State: Initialized, Role: FOLLOWER
I20260812 07:02:24.179185 13798 consensus_queue.cc:260] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b3c13e5fc549269a7da6a7a081160e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43921 } } peers { permanent_uuid: "c0e7b5e9592f4065bedee61976a1e02f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 45421 } } peers { permanent_uuid: "02f1fee168de40df9bf12e9424812f5a" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 39449 } }
I20260812 07:02:24.180709 13798 ts_tablet_manager.cc:1428] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e: Time spent starting tablet: real 0.011s user 0.005s sys 0.007s
I20260812 07:02:24.257699 13816 raft_consensus.cc:482] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:02:24.257987 13816 raft_consensus.cc:504] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b3c13e5fc549269a7da6a7a081160e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43921 } } peers { permanent_uuid: "c0e7b5e9592f4065bedee61976a1e02f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 45421 } } peers { permanent_uuid: "02f1fee168de40df9bf12e9424812f5a" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 39449 } }
I20260812 07:02:24.259632 13816 leader_election.cc:288] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0e7b5e9592f4065bedee61976a1e02f (127.11.192.195:45421), 02f1fee168de40df9bf12e9424812f5a (127.11.192.194:39449)
I20260812 07:02:24.266047 13792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:24.259943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:44837 (local address 127.11.192.195:45421)
0812 07:02:24.260174 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:24.260180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:24.260880 (+ 700us) server_negotiation.cc:407] Connection header received
0812 07:02:24.261078 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:24.261082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:24.261142 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:24.261214 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:24.261794 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.262535 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.264378 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.265143 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.265289 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:24.265364 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:24.265684 (+ 320us) server_negotiation.cc:299] Negotiation successful
0812 07:02:24.265808 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:02:24.266073 13790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:24.260146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:38147 (local address 127.11.192.194:39449)
0812 07:02:24.260319 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:24.260323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:24.260944 (+ 621us) server_negotiation.cc:407] Connection header received
0812 07:02:24.261109 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:24.261113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:24.261169 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:24.261241 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:24.261863 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.262553 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.264409 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.265167 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.265300 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:24.265372 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:24.265704 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 07:02:24.265825 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 07:02:24.266214 13818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:24.260160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:39449 (local address 127.11.192.193:38147)
0812 07:02:24.260843 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:24.260857 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:24.260979 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:24.261271 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:24.261276 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:24.261299 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:24.261709 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:24.261718 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.262685 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.262689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:24.264255 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:24.264269 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.265399 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.265405 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:24.265558 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:24.265580 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 07:02:24.265658 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 07:02:24.265773 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"mutex_wait_us":96,"thread_start_us":410,"threads_started":1}
I20260812 07:02:24.266463 13817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:24.259811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:45421 (local address 127.11.192.193:44837)
0812 07:02:24.260762 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:24.260780 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:24.260933 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:24.261244 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:24.261249 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:24.261272 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:24.261639 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:24.261652 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.262660 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.262665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:24.264227 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:24.264241 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.265375 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.265382 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:24.265535 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:24.265560 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:02:24.265638 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 07:02:24.265760 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"mutex_wait_us":7,"thread_start_us":561,"threads_started":1}
I20260812 07:02:24.267130 13452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b17868b8988e40f5ab89f72bcf5f7625" candidate_uuid: "f5b3c13e5fc549269a7da6a7a081160e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02f1fee168de40df9bf12e9424812f5a" is_pre_election: true
I20260812 07:02:24.267434 13690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b17868b8988e40f5ab89f72bcf5f7625" candidate_uuid: "f5b3c13e5fc549269a7da6a7a081160e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e7b5e9592f4065bedee61976a1e02f" is_pre_election: true
I20260812 07:02:24.267732 13452 raft_consensus.cc:2454] T b17868b8988e40f5ab89f72bcf5f7625 P 02f1fee168de40df9bf12e9424812f5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5b3c13e5fc549269a7da6a7a081160e in term 0.
I20260812 07:02:24.267953 13690 raft_consensus.cc:2454] T b17868b8988e40f5ab89f72bcf5f7625 P c0e7b5e9592f4065bedee61976a1e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5b3c13e5fc549269a7da6a7a081160e in term 0.
I20260812 07:02:24.268461 13109 leader_election.cc:302] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02f1fee168de40df9bf12e9424812f5a, f5b3c13e5fc549269a7da6a7a081160e; no voters:
I20260812 07:02:24.268821 13816 raft_consensus.cc:2781] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:02:24.268952 13816 raft_consensus.cc:482] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:02:24.269055 13816 raft_consensus.cc:3037] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:24.271237 13816 raft_consensus.cc:504] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b3c13e5fc549269a7da6a7a081160e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43921 } } peers { permanent_uuid: "c0e7b5e9592f4065bedee61976a1e02f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 45421 } } peers { permanent_uuid: "02f1fee168de40df9bf12e9424812f5a" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 39449 } }
I20260812 07:02:24.271823 13816 leader_election.cc:288] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e [CANDIDATE]: Term 1 election: Requested vote from peers c0e7b5e9592f4065bedee61976a1e02f (127.11.192.195:45421), 02f1fee168de40df9bf12e9424812f5a (127.11.192.194:39449)
I20260812 07:02:24.272220 13690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b17868b8988e40f5ab89f72bcf5f7625" candidate_uuid: "f5b3c13e5fc549269a7da6a7a081160e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e7b5e9592f4065bedee61976a1e02f"
I20260812 07:02:24.272311 13452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b17868b8988e40f5ab89f72bcf5f7625" candidate_uuid: "f5b3c13e5fc549269a7da6a7a081160e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02f1fee168de40df9bf12e9424812f5a"
I20260812 07:02:24.272379 13690 raft_consensus.cc:3037] T b17868b8988e40f5ab89f72bcf5f7625 P c0e7b5e9592f4065bedee61976a1e02f [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:24.272503 13452 raft_consensus.cc:3037] T b17868b8988e40f5ab89f72bcf5f7625 P 02f1fee168de40df9bf12e9424812f5a [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:24.275061 13690 raft_consensus.cc:2454] T b17868b8988e40f5ab89f72bcf5f7625 P c0e7b5e9592f4065bedee61976a1e02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5b3c13e5fc549269a7da6a7a081160e in term 1.
I20260812 07:02:24.275182 13452 raft_consensus.cc:2454] T b17868b8988e40f5ab89f72bcf5f7625 P 02f1fee168de40df9bf12e9424812f5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5b3c13e5fc549269a7da6a7a081160e in term 1.
I20260812 07:02:24.275473 13108 leader_election.cc:302] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c0e7b5e9592f4065bedee61976a1e02f, f5b3c13e5fc549269a7da6a7a081160e; no voters:
I20260812 07:02:24.275761 13816 raft_consensus.cc:2781] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:02:24.276211 13816 raft_consensus.cc:686] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e [term 1 LEADER]: Becoming Leader. State: Replica: f5b3c13e5fc549269a7da6a7a081160e, State: Running, Role: LEADER
I20260812 07:02:24.276695 13816 consensus_queue.cc:237] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b3c13e5fc549269a7da6a7a081160e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43921 } } peers { permanent_uuid: "c0e7b5e9592f4065bedee61976a1e02f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 45421 } } peers { permanent_uuid: "02f1fee168de40df9bf12e9424812f5a" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 39449 } }
I20260812 07:02:24.280944 12975 catalog_manager.cc:5495] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e reported cstate change: term changed from 0 to 1, leader changed from <none> to f5b3c13e5fc549269a7da6a7a081160e (127.11.192.193). New cstate: current_term: 1 leader_uuid: "f5b3c13e5fc549269a7da6a7a081160e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b3c13e5fc549269a7da6a7a081160e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43921 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0e7b5e9592f4065bedee61976a1e02f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 45421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02f1fee168de40df9bf12e9424812f5a" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 39449 } health_report { overall_health: UNKNOWN } } }
W20260812 07:02:24.291716 13766 tablet.cc:2369] T b17868b8988e40f5ab89f72bcf5f7625 P c0e7b5e9592f4065bedee61976a1e02f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 07:02:24 dist-test-slave-lgr5 krb5kdc[12907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.193@KRBTEST.COM
W20260812 07:02:24.323364 13541 tablet.cc:2369] T b17868b8988e40f5ab89f72bcf5f7625 P 02f1fee168de40df9bf12e9424812f5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:02:24.327737 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:24.313361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47610 (local address 127.11.192.193:43921)
0812 07:02:24.313545 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:24.313550 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:24.314024 (+ 474us) server_negotiation.cc:407] Connection header received
0812 07:02:24.314183 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:24.314186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:24.314249 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:24.314331 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:24.315949 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.316711 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.317980 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.318406 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.318513 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:24.322308 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:24.322325 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:24.322340 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:24.322378 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:24.325124 (+ 2746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:24.325752 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:24.325760 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:24.325762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:24.325924 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:24.326275 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:24.326279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:24.326281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:24.326627 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:24.326901 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:24.327194 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 07:02:24.327414 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260812 07:02:24.344974 13314 tablet.cc:2369] T b17868b8988e40f5ab89f72bcf5f7625 P f5b3c13e5fc549269a7da6a7a081160e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 07:02:24 dist-test-slave-lgr5 krb5kdc[12907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518144, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 07:02:24 dist-test-slave-lgr5 krb5kdc[12907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518144, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.192.193@KRBTEST.COM
I20260812 07:02:24.374245 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:24.361108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47618 (local address 127.11.192.193:43921)
0812 07:02:24.361288 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:24.361294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:24.361645 (+ 351us) server_negotiation.cc:407] Connection header received
0812 07:02:24.361815 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:24.361819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:24.361884 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:24.361979 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:24.363300 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.364022 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.365300 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.365705 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.365814 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:24.369375 (+ 3561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:24.369380 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:24.369383 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:24.369417 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:24.371775 (+ 2358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:24.372389 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:24.372394 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:24.372396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:24.372524 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:24.372898 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:24.372902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:24.372904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:24.373179 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:24.373453 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:24.373701 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 07:02:24.373917 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Aug 12 07:02:24 dist-test-slave-lgr5 krb5kdc[12907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 07:02:24 dist-test-slave-lgr5 krb5kdc[12907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.193@KRBTEST.COM
I20260812 07:02:24.419358 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:24.406319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47634 (local address 127.11.192.193:43921)
0812 07:02:24.406511 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:24.406518 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:24.406794 (+ 276us) server_negotiation.cc:407] Connection header received
0812 07:02:24.406966 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:24.406970 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:24.407062 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:24.407150 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:24.408469 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.409209 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.410452 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.410896 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.411014 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:24.414544 (+ 3530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:24.414549 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:24.414553 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:24.414586 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:24.416901 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:24.417481 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:24.417486 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:24.417488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:24.417617 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:24.418004 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:24.418008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:24.418020 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:24.418294 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:24.418534 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:24.418814 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 07:02:24.419054 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 07:02:24.421161 13153 tablet_service.cc:2908] Rejecting scan request for tablet b17868b8988e40f5ab89f72bcf5f7625: Uninitialized: clean time has not yet been initialized
I20260812 07:02:24.423909 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 13082
I20260812 07:02:24.434907 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:24.424720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47642 (local address 127.11.192.193:43921)
0812 07:02:24.424890 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:24.424896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:24.425140 (+ 244us) server_negotiation.cc:407] Connection header received
0812 07:02:24.425264 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:24.425267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:24.425321 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:24.425386 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:24.426408 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.427084 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.428198 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.428603 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.428688 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:24.430583 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:24.430588 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:24.430591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:24.430614 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:24.432791 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:24.433366 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:24.433369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:24.433371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:24.433480 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:24.433788 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:24.433790 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:24.433792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:24.434014 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:24.434182 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:24.434399 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 07:02:24.434583 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:02:24.435492 13303 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:24.524230 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 13082
I20260812 07:02:24.546566 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 13325
Aug 12 07:02:24 dist-test-slave-lgr5 krb5kdc[12907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.194@KRBTEST.COM
I20260812 07:02:24.560469 13790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:24.547432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49030 (local address 127.11.192.194:39449)
0812 07:02:24.547606 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:24.547610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:24.547629 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:02:24.547730 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:24.547734 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:24.547812 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:24.547900 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:24.549058 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.549808 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.551027 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.551401 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.551488 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:24.555429 (+ 3941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:24.555445 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:24.555460 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:24.555496 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:24.558128 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:24.558725 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:24.558732 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:24.558734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:24.558849 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:24.559121 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:24.559123 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:24.559125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:24.559500 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:24.559748 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:24.559922 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 07:02:24.560135 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:02:24.561209 13531 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:24.646545 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 13325
I20260812 07:02:24.673583 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 13547
Aug 12 07:02:24 dist-test-slave-lgr5 krb5kdc[12907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.195@KRBTEST.COM
I20260812 07:02:24.686990 13792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:24.674345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35678 (local address 127.11.192.195:45421)
0812 07:02:24.674527 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:24.674531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:24.674549 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:02:24.674680 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:24.674683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:24.674746 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:24.674839 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:24.675985 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.676740 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.677947 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:24.678326 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:24.678412 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:24.682393 (+ 3981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:24.682407 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:24.682424 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:24.682456 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:24.684871 (+ 2415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:24.685419 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:24.685425 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:24.685432 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:24.685560 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:24.685836 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:24.685838 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:24.685840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:24.686171 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:24.686380 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:24.686576 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 07:02:24.686734 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 07:02:24.687610 13754 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:24.773818 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 13547
I20260812 07:02:24.797497 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 12926
I20260812 07:02:24.798578 13026 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:24.905305 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 12926
2026-08-12T07:02:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4237 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:02:24 dist-test-slave-lgr5 krb5kdc[13847](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:02:24 dist-test-slave-lgr5 krb5kdc[13847](info): set up 2 sockets
Aug 12 07:02:24 dist-test-slave-lgr5 krb5kdc[13847](info): commencing operation
krb5kdc: starting...
W20260812 07:02:26.133980 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:02:26 dist-test-slave-lgr5 krb5kdc[13847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:02:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:02:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:02:26.312453 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:43965
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:43965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:26.622012 13867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:26.622273 13867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:26.622445 13867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:26.629452 13867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:02:26.629549 13867 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 07:02:26.629613 13867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:26.629700 13867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:26.629765 13867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:02:26.629848 13867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:02:26.639416 13867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:43965
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:43965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:26.640583 13867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:26.641842 13867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:26.652863 13878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:26.652810 13880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:26.653156 13885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:26.653981 13867 server_base.cc:1034] running on GCE node
I20260812 07:02:26.654677 13867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:26.655920 13867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:26.657153 13867 hybrid_clock.cc:648] HybridClock initialized: now 1786518146657079 us; error 42 us; skew 500 ppm
I20260812 07:02:26.657467 13867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:26 dist-test-slave-lgr5 krb5kdc[13847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518146, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:26.664193 13867 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:02:26.665506 13867 webserver.cc:460] Webserver started at http://127.11.192.254:46673/ using document root <none> and password file <none>
I20260812 07:02:26.666121 13867 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:26.666203 13867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:26.666456 13867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:26.669224 13867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "27f099ac496a47ca8f42644fd741339b"
format_stamp: "Formatted at 2026-08-12 07:02:26 on dist-test-slave-lgr5"
server_key: "adcdcedcadfb34ab1f110f75620eae1d"
server_key_iv: "36504965bf164b2fb7fe569eb01445ee"
server_key_version: "encryptionkey@0"
I20260812 07:02:26.669915 13867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "27f099ac496a47ca8f42644fd741339b"
format_stamp: "Formatted at 2026-08-12 07:02:26 on dist-test-slave-lgr5"
server_key: "adcdcedcadfb34ab1f110f75620eae1d"
server_key_iv: "36504965bf164b2fb7fe569eb01445ee"
server_key_version: "encryptionkey@0"
I20260812 07:02:26.674638 13867 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:02:26.677798 13893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:26.678982 13867 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 07:02:26.679121 13867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "27f099ac496a47ca8f42644fd741339b"
format_stamp: "Formatted at 2026-08-12 07:02:26 on dist-test-slave-lgr5"
server_key: "adcdcedcadfb34ab1f110f75620eae1d"
server_key_iv: "36504965bf164b2fb7fe569eb01445ee"
server_key_version: "encryptionkey@0"
I20260812 07:02:26.679306 13867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:26.712998 13867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:26.716775 13867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:26.717026 13867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:26.736701 13867 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:43965
I20260812 07:02:26.736733 13995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:43965 every 8 connection(s)
I20260812 07:02:26.737982 13867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:02:26.741539 13996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:26.747270 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 13867
I20260812 07:02:26.747481 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:02:26.748154 12035 external_mini_cluster.cc:1428] Setting key 87e7e4f687d11e81353b255f48248437
I20260812 07:02:26.751948 13996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b: Bootstrap starting.
I20260812 07:02:26.755439 13996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:26.756471 13996 log.cc:824] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:26.759380 13996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b: No bootstrap required, opened a new log
Aug 12 07:02:26 dist-test-slave-lgr5 krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:26.768360 13996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f099ac496a47ca8f42644fd741339b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43965 } }
I20260812 07:02:26.768725 13996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:26.768834 13996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f099ac496a47ca8f42644fd741339b, State: Initialized, Role: FOLLOWER
I20260812 07:02:26.769516 13996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f099ac496a47ca8f42644fd741339b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43965 } }
I20260812 07:02:26.769807 13996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:02:26.769950 13996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:02:26.770123 13996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:26.770828 14002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:26.750137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40696 (local address 127.11.192.254:43965)
0812 07:02:26.750915 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:26.750925 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:26.750966 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:02:26.752662 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:26.752684 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:26.753032 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:26.753363 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:26.755058 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:26.756424 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:26.757845 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:26.758552 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:26.758672 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:26.763224 (+ 4552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:26.763236 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:26.763253 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:26.763303 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:26.766955 (+ 3652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:26.767601 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:26.767613 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:26.767621 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:26.767778 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:26.768159 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:26.768165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:26.768168 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:26.768685 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:26.768986 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:26.769401 (+ 415us) server_negotiation.cc:299] Negotiation successful
0812 07:02:26.769720 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":344,"threads_started":1}
I20260812 07:02:26.772454 13996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f099ac496a47ca8f42644fd741339b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43965 } }
I20260812 07:02:26.772912 13996 leader_election.cc:302] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f099ac496a47ca8f42644fd741339b; no voters:
I20260812 07:02:26.773489 13996 leader_election.cc:288] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:02:26.773597 14004 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:02:26.774261 14004 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b [term 1 LEADER]: Becoming Leader. State: Replica: 27f099ac496a47ca8f42644fd741339b, State: Running, Role: LEADER
I20260812 07:02:26.774801 14004 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f099ac496a47ca8f42644fd741339b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43965 } }
I20260812 07:02:26.775472 13996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:02:26.777621 14006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27f099ac496a47ca8f42644fd741339b. Latest consensus state: current_term: 1 leader_uuid: "27f099ac496a47ca8f42644fd741339b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f099ac496a47ca8f42644fd741339b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43965 } } }
I20260812 07:02:26.778038 14006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:26.777644 14005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27f099ac496a47ca8f42644fd741339b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f099ac496a47ca8f42644fd741339b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43965 } } }
I20260812 07:02:26.778771 14005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:26.781797 14012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:02:26.787130 14012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:02:26.796563 14012 catalog_manager.cc:1347] Generated new cluster ID: 57f98929afde45ceb375cda9a2e69eab
I20260812 07:02:26.796662 14012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:02:26.812958 14012 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:02:26.814762 14012 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:02:26.828043 14012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b: Generated new TSK 0
I20260812 07:02:26.829113 14012 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
I20260812 07:02:26.873370 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43965
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:27.186661 14040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:27.186923 14040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:27.187057 14040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:27.193943 14040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:27.194042 14040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:27.194159 14040 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:02:27.194259 14040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:02:27.203475 14040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.192.254:43965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:27.204610 14040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:27.205864 14040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:27.218190 14061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:27.218192 14059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:27.218163 14057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:27.218973 14040 server_base.cc:1034] running on GCE node
I20260812 07:02:27.219576 14040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:27.220433 14040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:27.221642 14040 hybrid_clock.cc:648] HybridClock initialized: now 1786518147221574 us; error 43 us; skew 500 ppm
I20260812 07:02:27.221887 14040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:27 dist-test-slave-lgr5 krb5kdc[13847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518147, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:27.224874 14040 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:02:27.226212 14040 webserver.cc:460] Webserver started at http://127.11.192.193:38587/ using document root <none> and password file <none>
I20260812 07:02:27.226845 14040 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:27.226938 14040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:27.227252 14040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:27.230079 14040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "26f53c403df140bc8ce7e43c59dcb10b"
format_stamp: "Formatted at 2026-08-12 07:02:27 on dist-test-slave-lgr5"
server_key: "93e664c79928cb037aa4c0551deb1390"
server_key_iv: "1a8a9de82738f56fe6f1a79a9ca7513f"
server_key_version: "encryptionkey@0"
I20260812 07:02:27.230845 14040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "26f53c403df140bc8ce7e43c59dcb10b"
format_stamp: "Formatted at 2026-08-12 07:02:27 on dist-test-slave-lgr5"
server_key: "93e664c79928cb037aa4c0551deb1390"
server_key_iv: "1a8a9de82738f56fe6f1a79a9ca7513f"
server_key_version: "encryptionkey@0"
I20260812 07:02:27.235743 14040 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260812 07:02:27.239085 14075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:27.240345 14040 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 07:02:27.240481 14040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "26f53c403df140bc8ce7e43c59dcb10b"
format_stamp: "Formatted at 2026-08-12 07:02:27 on dist-test-slave-lgr5"
server_key: "93e664c79928cb037aa4c0551deb1390"
server_key_iv: "1a8a9de82738f56fe6f1a79a9ca7513f"
server_key_version: "encryptionkey@0"
I20260812 07:02:27.240671 14040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:27.260262 14040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:27.264214 14040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:27.264477 14040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:27.267206 14040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:27.267292 14040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:27.267434 14040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:27.267532 14040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:02:27 dist-test-slave-lgr5 krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518147, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:27.289426 14002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:27.267755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:52937 (local address 127.11.192.254:43965)
0812 07:02:27.267955 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:27.267962 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:27.269258 (+ 1296us) server_negotiation.cc:407] Connection header received
0812 07:02:27.271193 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:27.271199 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:27.271286 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:27.271385 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:27.273184 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:27.274128 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:27.276178 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:27.276769 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:27.276891 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:27.281814 (+ 4923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:27.281828 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:27.281834 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:27.281891 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:27.285544 (+ 3653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:27.286357 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:27.286364 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:27.286367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:27.286542 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:27.287262 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:27.287269 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:27.287274 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:27.287676 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:27.287850 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:27.288720 (+ 870us) server_negotiation.cc:299] Negotiation successful
0812 07:02:27.288977 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 07:02:27.290103 14094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:27.268241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43965 (local address 127.11.192.193:52937)
0812 07:02:27.269080 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:27.269123 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:27.270937 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:27.271620 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:27.271627 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:27.272187 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:27.273000 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:27.273029 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:27.274326 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:27.274333 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:27.276022 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:27.276034 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:27.277009 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:27.277017 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:27.277327 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:27.278024 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:27.278066 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:27.281475 (+ 3409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:27.285720 (+ 4245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:27.285729 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:27.285743 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:27.286197 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:27.286732 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:27.286739 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:27.286744 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:27.287079 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:27.287827 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:27.287835 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:27.288288 (+ 453us) client_negotiation.cc:769] Sending connection context
0812 07:02:27.288584 (+ 296us) client_negotiation.cc:240] Negotiation successful
0812 07:02:27.288988 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":348,"threads_started":1}
I20260812 07:02:27.311232 14196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:27.295722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43965 (local address 127.11.192.193:55567)
0812 07:02:27.296368 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:27.296387 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:27.296517 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:27.296801 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:27.296809 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:27.297436 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:27.297857 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:27.297873 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:27.299298 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:27.299303 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:27.301400 (+ 2097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:27.301420 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:27.302280 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:27.302288 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:27.302499 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:27.303079 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:27.303107 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:27.304209 (+ 1102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:27.307894 (+ 3685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:27.307904 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:27.307908 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:27.308659 (+ 751us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:27.309195 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:27.309201 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:27.309208 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:27.309395 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:27.310040 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:27.310045 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:27.310237 (+ 192us) client_negotiation.cc:769] Sending connection context
0812 07:02:27.310412 (+ 175us) client_negotiation.cc:240] Negotiation successful
0812 07:02:27.310721 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":323,"threads_started":1}
I20260812 07:02:27.311990 14002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:27.295709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:55567 (local address 127.11.192.254:43965)
0812 07:02:27.295888 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:27.295895 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:27.296464 (+ 569us) server_negotiation.cc:407] Connection header received
0812 07:02:27.296620 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:27.296625 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:27.296682 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:27.296757 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:27.298015 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:27.299124 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:27.301580 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:27.302046 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:27.302168 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:27.304474 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:27.304481 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:27.304487 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:27.304529 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:27.307702 (+ 3173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:27.308872 (+ 1170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:27.308879 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:27.308883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:27.309044 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:27.309580 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:27.309584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:27.309586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:27.309924 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:27.310058 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:27.311536 (+ 1478us) server_negotiation.cc:299] Negotiation successful
0812 07:02:27.311700 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:02:27.314263 14040 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:38347
I20260812 07:02:27.314286 14286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:38347 every 8 connection(s)
I20260812 07:02:27.315609 14040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:02:27.317549 14288 heartbeater.cc:347] Connected to a master server at 127.11.192.254:43965
I20260812 07:02:27.317757 14288 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:27.318378 14288 heartbeater.cc:511] Master 127.11.192.254:43965 requested a full tablet report, sending...
I20260812 07:02:27.320057 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 14040
I20260812 07:02:27.320334 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:02:27.320439 13921 ts_manager.cc:194] Registered new tserver with Master: 26f53c403df140bc8ce7e43c59dcb10b (127.11.192.193:38347)
I20260812 07:02:27.321137 12035 external_mini_cluster.cc:1428] Setting key b9cc4eedb302e129508eea7f37c139ba
I20260812 07:02:27.322284 13921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:52937
WARNING: no policy specified for kudu/127.11.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
I20260812 07:02:27.360903 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43965
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:27.672343 14293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:27.672590 14293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:27.672719 14293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:27.679597 14293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:27.679692 14293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:27.679802 14293 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:02:27.679904 14293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:02:27.689286 14293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.192.254:43965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:27.690433 14293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:27.691733 14293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:27.703996 14301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:27.704107 14303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:27.704342 14300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:27.704931 14293 server_base.cc:1034] running on GCE node
I20260812 07:02:27.705466 14293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:27.706295 14293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:27.707509 14293 hybrid_clock.cc:648] HybridClock initialized: now 1786518147707434 us; error 45 us; skew 500 ppm
I20260812 07:02:27.707774 14293 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:27 dist-test-slave-lgr5 krb5kdc[13847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518147, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:27.710680 14293 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:02:27.711961 14293 webserver.cc:460] Webserver started at http://127.11.192.194:38909/ using document root <none> and password file <none>
I20260812 07:02:27.712647 14293 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:27.712750 14293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:27.713059 14293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:27.715782 14293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "74190da3ddc84b1dbeb079b114c55852"
format_stamp: "Formatted at 2026-08-12 07:02:27 on dist-test-slave-lgr5"
server_key: "e3e781825ee06848582334a940598dba"
server_key_iv: "90c3960d5dc62f35a7c0dfca73e251fc"
server_key_version: "encryptionkey@0"
I20260812 07:02:27.716567 14293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "74190da3ddc84b1dbeb079b114c55852"
format_stamp: "Formatted at 2026-08-12 07:02:27 on dist-test-slave-lgr5"
server_key: "e3e781825ee06848582334a940598dba"
server_key_iv: "90c3960d5dc62f35a7c0dfca73e251fc"
server_key_version: "encryptionkey@0"
I20260812 07:02:27.721356 14293 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:02:27.724576 14315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:27.725764 14293 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:02:27.725916 14293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "74190da3ddc84b1dbeb079b114c55852"
format_stamp: "Formatted at 2026-08-12 07:02:27 on dist-test-slave-lgr5"
server_key: "e3e781825ee06848582334a940598dba"
server_key_iv: "90c3960d5dc62f35a7c0dfca73e251fc"
server_key_version: "encryptionkey@0"
I20260812 07:02:27.726109 14293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:27.741562 14293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:27.745397 14293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:27.745640 14293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:27.748346 14293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:27.748483 14293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:27.748623 14293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:27.748694 14293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:02:27 dist-test-slave-lgr5 krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518147, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:27.770902 14328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:27.749713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43965 (local address 127.11.192.194:53887)
0812 07:02:27.750580 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:27.750632 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:27.752607 (+ 1975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:27.753301 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:27.753308 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:27.753906 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:27.754579 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:27.754600 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:27.756036 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:27.756042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:27.758653 (+ 2611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:27.758669 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:27.759576 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:27.759589 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:27.759786 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:27.760414 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:27.760448 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:27.763187 (+ 2739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:27.766109 (+ 2922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:27.766117 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:27.766132 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:27.766684 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:27.767121 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:27.767125 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:27.767127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:27.767371 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:27.768113 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:27.768124 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:27.768593 (+ 469us) client_negotiation.cc:769] Sending connection context
0812 07:02:27.768887 (+ 294us) client_negotiation.cc:240] Negotiation successful
0812 07:02:27.769426 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":374,"threads_started":1}
I20260812 07:02:27.772292 14002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:27.749073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:53887 (local address 127.11.192.254:43965)
0812 07:02:27.749229 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:27.749235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:27.750733 (+ 1498us) server_negotiation.cc:407] Connection header received
0812 07:02:27.752869 (+ 2136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:27.752874 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:27.752961 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:27.753057 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:27.754796 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:27.755817 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:27.758810 (+ 2993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:27.759343 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:27.759472 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:27.763392 (+ 3920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:27.763397 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:27.763400 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:27.763432 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:27.765972 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:27.766834 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:27.766840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:27.766844 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:27.766999 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:27.767500 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:27.767506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:27.767510 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:27.767883 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:27.768045 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:27.771662 (+ 3617us) server_negotiation.cc:299] Negotiation successful
0812 07:02:27.771924 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 07:02:27.791414 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:27.778666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43965 (local address 127.11.192.194:52025)
0812 07:02:27.779307 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:27.779329 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:27.779469 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:27.779740 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:27.779745 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:27.780398 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:27.780889 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:27.780904 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:27.782156 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:27.782164 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:27.783677 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:27.783689 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:27.784501 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:27.784508 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:27.784690 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:27.785325 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:27.785341 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:27.786028 (+ 687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:27.789166 (+ 3138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:27.789171 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:27.789173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:27.789539 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:27.789873 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:27.789877 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:27.789880 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:27.790003 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:27.790515 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:27.790519 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:27.790682 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 07:02:27.790822 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 07:02:27.791119 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":336,"threads_started":1}
I20260812 07:02:27.792287 14002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:27.778561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:52025 (local address 127.11.192.254:43965)
0812 07:02:27.778736 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:27.778742 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:27.779416 (+ 674us) server_negotiation.cc:407] Connection header received
0812 07:02:27.779565 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:27.779568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:27.779624 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:27.779696 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:27.781064 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:27.782030 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:27.783814 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:27.784289 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:27.784393 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:27.786202 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:27.786208 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:27.786213 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:27.786251 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:27.789040 (+ 2789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:27.789644 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:27.789649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:27.789653 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:27.789772 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:27.790102 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:27.790107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:27.790111 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:27.790401 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:27.790538 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:27.791816 (+ 1278us) server_negotiation.cc:299] Negotiation successful
0812 07:02:27.792021 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 07:02:27.800603 14293 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:37823
I20260812 07:02:27.800624 14523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:37823 every 8 connection(s)
I20260812 07:02:27.801692 14293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:02:27.803402 14526 heartbeater.cc:347] Connected to a master server at 127.11.192.254:43965
I20260812 07:02:27.803607 14526 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:27.804220 14526 heartbeater.cc:511] Master 127.11.192.254:43965 requested a full tablet report, sending...
I20260812 07:02:27.805399 13921 ts_manager.cc:194] Registered new tserver with Master: 74190da3ddc84b1dbeb079b114c55852 (127.11.192.194:37823)
I20260812 07:02:27.805881 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 14293
I20260812 07:02:27.806061 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:02:27.806258 13921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:53887
I20260812 07:02:27.806489 12035 external_mini_cluster.cc:1428] Setting key c9cdaba874ca426272091e836a73a790
WARNING: no policy specified for kudu/127.11.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
I20260812 07:02:27.846832 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43965
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:28.165542 14535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:28.165807 14535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:28.165946 14535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:28.172902 14535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:28.172998 14535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:28.173116 14535 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:02:28.173216 14535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:02:28.182719 14535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.192.254:43965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:28.184036 14535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:28.185273 14535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:28.197532 14550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:28.197566 14547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:28.197933 14552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:28.198326 14535 server_base.cc:1034] running on GCE node
I20260812 07:02:28.198875 14535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:28.199688 14535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:28.200897 14535 hybrid_clock.cc:648] HybridClock initialized: now 1786518148200832 us; error 41 us; skew 500 ppm
I20260812 07:02:28.201169 14535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:28 dist-test-slave-lgr5 krb5kdc[13847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518148, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:28.204161 14535 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:02:28.205569 14535 webserver.cc:460] Webserver started at http://127.11.192.195:46679/ using document root <none> and password file <none>
I20260812 07:02:28.206175 14535 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:28.206259 14535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:28.206528 14535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:28.209236 14535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "6e2311c83d1044b3afd795ce6e02ce49"
format_stamp: "Formatted at 2026-08-12 07:02:28 on dist-test-slave-lgr5"
server_key: "c0d0c29d4511eb501a816ab1e67c3aac"
server_key_iv: "86b8db6575dd8182c35a3b2022919c36"
server_key_version: "encryptionkey@0"
I20260812 07:02:28.209959 14535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "6e2311c83d1044b3afd795ce6e02ce49"
format_stamp: "Formatted at 2026-08-12 07:02:28 on dist-test-slave-lgr5"
server_key: "c0d0c29d4511eb501a816ab1e67c3aac"
server_key_iv: "86b8db6575dd8182c35a3b2022919c36"
server_key_version: "encryptionkey@0"
I20260812 07:02:28.214685 14535 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:02:28.217897 14562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:28.219094 14535 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 07:02:28.219241 14535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "6e2311c83d1044b3afd795ce6e02ce49"
format_stamp: "Formatted at 2026-08-12 07:02:28 on dist-test-slave-lgr5"
server_key: "c0d0c29d4511eb501a816ab1e67c3aac"
server_key_iv: "86b8db6575dd8182c35a3b2022919c36"
server_key_version: "encryptionkey@0"
I20260812 07:02:28.219449 14535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:28.245350 14535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:28.249131 14535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:28.249397 14535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:28.252162 14535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:28.252276 14535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260812 07:02:28.252430 14535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:28.252494 14535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:02:28 dist-test-slave-lgr5 krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518148, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:28.275460 14002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.252487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:44899 (local address 127.11.192.254:43965)
0812 07:02:28.252666 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:28.252672 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:28.254151 (+ 1479us) server_negotiation.cc:407] Connection header received
0812 07:02:28.256162 (+ 2011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:28.256168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:28.256252 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:28.256341 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:28.258389 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.259340 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.261955 (+ 2615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.262598 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.262743 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.269522 (+ 6779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:28.269529 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:28.269534 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:28.269571 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:28.272009 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:28.272981 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:28.272987 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:28.272991 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:28.273168 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:28.273653 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:28.273658 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:28.273661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:28.273974 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:28.274113 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:28.274821 (+ 708us) server_negotiation.cc:299] Negotiation successful
0812 07:02:28.275089 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:02:28.276748 14574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.253210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43965 (local address 127.11.192.195:44899)
0812 07:02:28.253997 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:28.254033 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:28.255856 (+ 1823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:28.256576 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:28.256584 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:28.257277 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:28.258138 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:28.258168 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.259520 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.259526 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:28.261793 (+ 2267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:28.261813 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.262840 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.262849 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:28.263179 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.266096 (+ 2917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:28.266122 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:28.269292 (+ 3170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:28.272163 (+ 2871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:28.272171 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:28.272186 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:28.272819 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:28.273306 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:28.273309 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:28.273312 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:28.273535 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:28.274095 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:28.274102 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:28.274424 (+ 322us) client_negotiation.cc:769] Sending connection context
0812 07:02:28.274701 (+ 277us) client_negotiation.cc:240] Negotiation successful
0812 07:02:28.275226 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":336,"threads_started":1}
I20260812 07:02:28.298103 14535 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:33171
I20260812 07:02:28.298128 14767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:33171 every 8 connection(s)
I20260812 07:02:28.299643 14703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.284837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43965 (local address 127.11.192.195:52763)
0812 07:02:28.285558 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:28.285579 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:28.285798 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:28.286394 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:28.286398 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:28.287164 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:28.287712 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:28.287730 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.289140 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.289143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:28.290571 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:28.290589 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.291574 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.291578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:28.291767 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.292484 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:28.292512 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:28.293289 (+ 777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:28.297112 (+ 3823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:28.297117 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:28.297121 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:28.297693 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:28.298158 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:28.298163 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:28.298167 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:28.298325 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:28.298888 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:28.298891 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:28.299038 (+ 147us) client_negotiation.cc:769] Sending connection context
0812 07:02:28.299145 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 07:02:28.299362 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":583,"threads_started":1}
I20260812 07:02:28.300031 14002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.285249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:52763 (local address 127.11.192.254:43965)
0812 07:02:28.285432 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:28.285437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:28.285674 (+ 237us) server_negotiation.cc:407] Connection header received
0812 07:02:28.286013 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:28.286019 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:28.286099 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:28.286171 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:28.287877 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.289008 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.290735 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.291383 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.291510 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.293472 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:28.293479 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:28.293484 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:28.293523 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:28.296957 (+ 3434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:28.297826 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:28.297832 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:28.297836 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:28.298038 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:28.298454 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:28.298459 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:28.298462 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:28.298788 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:28.298938 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:28.299294 (+ 356us) server_negotiation.cc:299] Negotiation successful
0812 07:02:28.299496 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:02:28.300042 14535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:02:28.302126 14768 heartbeater.cc:347] Connected to a master server at 127.11.192.254:43965
I20260812 07:02:28.303123 14768 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:28.303742 14768 heartbeater.cc:511] Master 127.11.192.254:43965 requested a full tablet report, sending...
I20260812 07:02:28.304988 13921 ts_manager.cc:194] Registered new tserver with Master: 6e2311c83d1044b3afd795ce6e02ce49 (127.11.192.195:33171)
I20260812 07:02:28.305719 13921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:44899
I20260812 07:02:28.306555 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 14535
I20260812 07:02:28.306728 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:02:28.307150 12035 external_mini_cluster.cc:1428] Setting key eafae8b76f3bc17a30ab409bcc561086
I20260812 07:02:28.309765 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:02:28.324940 14288 heartbeater.cc:503] Master 127.11.192.254:43965 was elected leader, sending a full tablet report...
Aug 12 07:02:28 dist-test-slave-lgr5 krb5kdc[13847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518148, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 07:02:28 dist-test-slave-lgr5 krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518148, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:28.357686 14002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.345278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40706 (local address 127.11.192.254:43965)
0812 07:02:28.345517 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:28.345524 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:28.345756 (+ 232us) server_negotiation.cc:407] Connection header received
0812 07:02:28.345917 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:28.345921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:28.345984 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:28.346065 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:28.347296 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.348013 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.349220 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.349653 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.349766 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.353348 (+ 3582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:28.353353 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:28.353356 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:28.353386 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:28.355579 (+ 2193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:28.356163 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:28.356167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:28.356169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:28.356292 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:28.356575 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:28.356578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:28.356580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:28.356824 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:28.356993 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:28.357210 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 07:02:28.357374 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 07:02:28.361399 13921 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:02:28.368067 13921 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:02:28.399654 14800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.379370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:37823 (local address 127.0.0.1:42170)
0812 07:02:28.391394 (+ 12024us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:28.391412 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:28.391528 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:28.392331 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:28.392336 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:28.392360 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:28.392826 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:28.392841 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.394581 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.394587 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:28.397334 (+ 2747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:28.397350 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.398633 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.398639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:28.398852 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.398891 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 07:02:28.398988 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 07:02:28.399173 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11890,"mutex_wait_us":258,"thread_start_us":12393,"threads_started":1}
I20260812 07:02:28.400621 14794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.379321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:38347 (local address 127.0.0.1:35854)
0812 07:02:28.390561 (+ 11240us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:28.390599 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:28.390765 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:28.391512 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:28.391518 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:28.391550 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:28.393028 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:28.393042 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.395014 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.395019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:28.397593 (+ 2574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:28.397605 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.399623 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.399629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:28.399965 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.400012 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 07:02:28.400122 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 07:02:28.400249 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11057,"mutex_wait_us":34,"thread_start_us":10942,"threads_started":1}
I20260812 07:02:28.400662 14795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.379406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35854 (local address 127.11.192.193:38347)
0812 07:02:28.390223 (+ 10817us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:28.390232 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:28.390710 (+ 478us) server_negotiation.cc:407] Connection header received
0812 07:02:28.390992 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:28.391000 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:28.391262 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:28.391572 (+ 310us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:28.393203 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.394621 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.397765 (+ 3144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.399157 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.399321 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.399469 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:28.400211 (+ 742us) server_negotiation.cc:299] Negotiation successful
0812 07:02:28.400382 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10668,"thread_start_us":10542,"threads_started":1}
I20260812 07:02:28.400696 14796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.379669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42170 (local address 127.11.192.194:37823)
0812 07:02:28.389656 (+ 9987us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:28.389666 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:28.391497 (+ 1831us) server_negotiation.cc:407] Connection header received
0812 07:02:28.391682 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:28.391690 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:28.391908 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:28.392124 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:28.392996 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.394457 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.397499 (+ 3042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.398436 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.398594 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.398772 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:28.400287 (+ 1515us) server_negotiation.cc:299] Negotiation successful
0812 07:02:28.400455 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9792,"thread_start_us":9960,"threads_started":1}
I20260812 07:02:28.402202 14790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.380795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:33171 (local address 127.0.0.1:54598)
0812 07:02:28.392018 (+ 11223us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:28.392035 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:28.392153 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:28.395541 (+ 3388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:28.395545 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:28.395573 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:28.396136 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:28.396149 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.397676 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.397681 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:28.399698 (+ 2017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:28.399711 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.401477 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.401483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:28.401652 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.401684 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 07:02:28.401776 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 07:02:28.401908 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11107,"thread_start_us":10326,"threads_started":1}
I20260812 07:02:28.402788 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.381481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54598 (local address 127.11.192.195:33171)
0812 07:02:28.395109 (+ 13628us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:28.395117 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:28.395146 (+ 29us) server_negotiation.cc:407] Connection header received
0812 07:02:28.395217 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:28.395222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:28.395371 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:28.395521 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:28.396463 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.397546 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.399875 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.401284 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.401494 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.401654 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:28.401850 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 07:02:28.402010 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13442,"thread_start_us":13263,"threads_started":1}
I20260812 07:02:28.404281 14401 tablet_service.cc:1467] Processing CreateTablet for tablet c5dafb0ef6394dc1af1845c511be13a6 (DEFAULT_TABLE table=test-table [id=fa2053ee6f3e4261838dd0ebfdb43e69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:28.404407 14156 tablet_service.cc:1467] Processing CreateTablet for tablet c5dafb0ef6394dc1af1845c511be13a6 (DEFAULT_TABLE table=test-table [id=fa2053ee6f3e4261838dd0ebfdb43e69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:28.405432 14156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5dafb0ef6394dc1af1845c511be13a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:28.405433 14401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5dafb0ef6394dc1af1845c511be13a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:28.411705 14645 tablet_service.cc:1467] Processing CreateTablet for tablet c5dafb0ef6394dc1af1845c511be13a6 (DEFAULT_TABLE table=test-table [id=fa2053ee6f3e4261838dd0ebfdb43e69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:28.413108 14645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5dafb0ef6394dc1af1845c511be13a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:28.416311 14806 tablet_bootstrap.cc:492] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852: Bootstrap starting.
I20260812 07:02:28.419984 14806 tablet_bootstrap.cc:654] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:28.420819 14808 tablet_bootstrap.cc:492] T c5dafb0ef6394dc1af1845c511be13a6 P 26f53c403df140bc8ce7e43c59dcb10b: Bootstrap starting.
I20260812 07:02:28.421290 14806 log.cc:824] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:28.424674 14808 tablet_bootstrap.cc:654] T c5dafb0ef6394dc1af1845c511be13a6 P 26f53c403df140bc8ce7e43c59dcb10b: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:28.425905 14808 log.cc:824] T c5dafb0ef6394dc1af1845c511be13a6 P 26f53c403df140bc8ce7e43c59dcb10b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:28.428025 14812 tablet_bootstrap.cc:492] T c5dafb0ef6394dc1af1845c511be13a6 P 6e2311c83d1044b3afd795ce6e02ce49: Bootstrap starting.
I20260812 07:02:28.430692 14806 tablet_bootstrap.cc:492] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852: No bootstrap required, opened a new log
I20260812 07:02:28.430792 14808 tablet_bootstrap.cc:492] T c5dafb0ef6394dc1af1845c511be13a6 P 26f53c403df140bc8ce7e43c59dcb10b: No bootstrap required, opened a new log
I20260812 07:02:28.431051 14808 ts_tablet_manager.cc:1397] T c5dafb0ef6394dc1af1845c511be13a6 P 26f53c403df140bc8ce7e43c59dcb10b: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260812 07:02:28.431051 14806 ts_tablet_manager.cc:1397] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.004s
I20260812 07:02:28.431442 14812 tablet_bootstrap.cc:654] T c5dafb0ef6394dc1af1845c511be13a6 P 6e2311c83d1044b3afd795ce6e02ce49: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:28.432451 14812 log.cc:824] T c5dafb0ef6394dc1af1845c511be13a6 P 6e2311c83d1044b3afd795ce6e02ce49: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:28.434706 14812 tablet_bootstrap.cc:492] T c5dafb0ef6394dc1af1845c511be13a6 P 6e2311c83d1044b3afd795ce6e02ce49: No bootstrap required, opened a new log
I20260812 07:02:28.434927 14812 ts_tablet_manager.cc:1397] T c5dafb0ef6394dc1af1845c511be13a6 P 6e2311c83d1044b3afd795ce6e02ce49: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 07:02:28.441416 14806 raft_consensus.cc:348] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } }
I20260812 07:02:28.441411 14808 raft_consensus.cc:348] T c5dafb0ef6394dc1af1845c511be13a6 P 26f53c403df140bc8ce7e43c59dcb10b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } }
I20260812 07:02:28.441855 14808 raft_consensus.cc:374] T c5dafb0ef6394dc1af1845c511be13a6 P 26f53c403df140bc8ce7e43c59dcb10b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:28.441846 14806 raft_consensus.cc:374] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:28.441955 14806 raft_consensus.cc:729] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74190da3ddc84b1dbeb079b114c55852, State: Initialized, Role: FOLLOWER
I20260812 07:02:28.441998 14808 raft_consensus.cc:729] T c5dafb0ef6394dc1af1845c511be13a6 P 26f53c403df140bc8ce7e43c59dcb10b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26f53c403df140bc8ce7e43c59dcb10b, State: Initialized, Role: FOLLOWER
I20260812 07:02:28.441839 14812 raft_consensus.cc:348] T c5dafb0ef6394dc1af1845c511be13a6 P 6e2311c83d1044b3afd795ce6e02ce49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } }
I20260812 07:02:28.442282 14812 raft_consensus.cc:374] T c5dafb0ef6394dc1af1845c511be13a6 P 6e2311c83d1044b3afd795ce6e02ce49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:28.442416 14812 raft_consensus.cc:729] T c5dafb0ef6394dc1af1845c511be13a6 P 6e2311c83d1044b3afd795ce6e02ce49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e2311c83d1044b3afd795ce6e02ce49, State: Initialized, Role: FOLLOWER
I20260812 07:02:28.442581 14806 consensus_queue.cc:260] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } }
I20260812 07:02:28.442580 14808 consensus_queue.cc:260] T c5dafb0ef6394dc1af1845c511be13a6 P 26f53c403df140bc8ce7e43c59dcb10b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } }
I20260812 07:02:28.443019 14812 consensus_queue.cc:260] T c5dafb0ef6394dc1af1845c511be13a6 P 6e2311c83d1044b3afd795ce6e02ce49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } }
I20260812 07:02:28.444404 14808 ts_tablet_manager.cc:1428] T c5dafb0ef6394dc1af1845c511be13a6 P 26f53c403df140bc8ce7e43c59dcb10b: Time spent starting tablet: real 0.013s user 0.004s sys 0.010s
I20260812 07:02:28.444404 14806 ts_tablet_manager.cc:1428] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852: Time spent starting tablet: real 0.013s user 0.005s sys 0.008s
I20260812 07:02:28.444967 14526 heartbeater.cc:503] Master 127.11.192.254:43965 was elected leader, sending a full tablet report...
I20260812 07:02:28.447856 14768 heartbeater.cc:503] Master 127.11.192.254:43965 was elected leader, sending a full tablet report...
I20260812 07:02:28.448572 14812 ts_tablet_manager.cc:1428] T c5dafb0ef6394dc1af1845c511be13a6 P 6e2311c83d1044b3afd795ce6e02ce49: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260812 07:02:28.468869 14818 raft_consensus.cc:482] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:02:28.469165 14818 raft_consensus.cc:504] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } }
I20260812 07:02:28.470853 14818 leader_election.cc:288] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 26f53c403df140bc8ce7e43c59dcb10b (127.11.192.193:38347), 6e2311c83d1044b3afd795ce6e02ce49 (127.11.192.195:33171)
I20260812 07:02:28.477631 14795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.471273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:49271 (local address 127.11.192.193:38347)
0812 07:02:28.471479 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:28.471486 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:28.472298 (+ 812us) server_negotiation.cc:407] Connection header received
0812 07:02:28.472469 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:28.472473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:28.472529 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:28.472604 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:28.473221 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.474017 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.475875 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.476715 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.476857 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.476937 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:28.477294 (+ 357us) server_negotiation.cc:299] Negotiation successful
0812 07:02:28.477412 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:02:28.477634 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.471321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:54259 (local address 127.11.192.195:33171)
0812 07:02:28.471556 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:28.471563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:28.472179 (+ 616us) server_negotiation.cc:407] Connection header received
0812 07:02:28.472375 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:28.472379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:28.472436 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:28.472514 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:28.473076 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.473793 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.475803 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.476705 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.476857 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.476934 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:28.477276 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 07:02:28.477398 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260812 07:02:28.477783 14825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.471130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:33171 (local address 127.11.192.194:54259)
0812 07:02:28.472053 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:28.472068 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:28.472239 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:28.472535 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:28.472539 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:28.472562 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:28.472921 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:28.472935 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.473919 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.473925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:28.475628 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:28.475645 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.476937 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.476945 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:28.477084 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.477145 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 07:02:28.477228 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 07:02:28.477356 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"mutex_wait_us":22,"thread_start_us":930,"threads_started":1}
I20260812 07:02:28.477908 14824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.471206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:38347 (local address 127.11.192.194:49271)
0812 07:02:28.472194 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:28.472211 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:28.472336 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:28.472634 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:28.472638 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:28.472677 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:28.473075 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:28.473084 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.474141 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.474146 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:28.475719 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:28.475731 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.476982 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.476987 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:28.477146 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.477171 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:02:28.477250 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 07:02:28.477379 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":844,"mutex_wait_us":17,"thread_start_us":455,"threads_started":1}
I20260812 07:02:28.478895 14686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5dafb0ef6394dc1af1845c511be13a6" candidate_uuid: "74190da3ddc84b1dbeb079b114c55852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2311c83d1044b3afd795ce6e02ce49" is_pre_election: true
I20260812 07:02:28.479067 14203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5dafb0ef6394dc1af1845c511be13a6" candidate_uuid: "74190da3ddc84b1dbeb079b114c55852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26f53c403df140bc8ce7e43c59dcb10b" is_pre_election: true
I20260812 07:02:28.479408 14686 raft_consensus.cc:2454] T c5dafb0ef6394dc1af1845c511be13a6 P 6e2311c83d1044b3afd795ce6e02ce49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74190da3ddc84b1dbeb079b114c55852 in term 0.
I20260812 07:02:28.479650 14203 raft_consensus.cc:2454] T c5dafb0ef6394dc1af1845c511be13a6 P 26f53c403df140bc8ce7e43c59dcb10b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74190da3ddc84b1dbeb079b114c55852 in term 0.
I20260812 07:02:28.480042 14317 leader_election.cc:302] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2311c83d1044b3afd795ce6e02ce49, 74190da3ddc84b1dbeb079b114c55852; no voters:
I20260812 07:02:28.480486 14818 raft_consensus.cc:2781] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:02:28.480607 14818 raft_consensus.cc:482] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:02:28.480685 14818 raft_consensus.cc:3037] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:28.482776 14818 raft_consensus.cc:504] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } }
I20260812 07:02:28.483356 14818 leader_election.cc:288] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852 [CANDIDATE]: Term 1 election: Requested vote from peers 26f53c403df140bc8ce7e43c59dcb10b (127.11.192.193:38347), 6e2311c83d1044b3afd795ce6e02ce49 (127.11.192.195:33171)
I20260812 07:02:28.483664 14203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5dafb0ef6394dc1af1845c511be13a6" candidate_uuid: "74190da3ddc84b1dbeb079b114c55852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26f53c403df140bc8ce7e43c59dcb10b"
I20260812 07:02:28.483857 14203 raft_consensus.cc:3037] T c5dafb0ef6394dc1af1845c511be13a6 P 26f53c403df140bc8ce7e43c59dcb10b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:28.483872 14686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5dafb0ef6394dc1af1845c511be13a6" candidate_uuid: "74190da3ddc84b1dbeb079b114c55852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2311c83d1044b3afd795ce6e02ce49"
I20260812 07:02:28.484057 14686 raft_consensus.cc:3037] T c5dafb0ef6394dc1af1845c511be13a6 P 6e2311c83d1044b3afd795ce6e02ce49 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:28.487007 14203 raft_consensus.cc:2454] T c5dafb0ef6394dc1af1845c511be13a6 P 26f53c403df140bc8ce7e43c59dcb10b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74190da3ddc84b1dbeb079b114c55852 in term 1.
I20260812 07:02:28.487076 14686 raft_consensus.cc:2454] T c5dafb0ef6394dc1af1845c511be13a6 P 6e2311c83d1044b3afd795ce6e02ce49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74190da3ddc84b1dbeb079b114c55852 in term 1.
I20260812 07:02:28.487416 14318 leader_election.cc:302] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26f53c403df140bc8ce7e43c59dcb10b, 74190da3ddc84b1dbeb079b114c55852; no voters:
I20260812 07:02:28.487691 14818 raft_consensus.cc:2781] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:02:28.488145 14818 raft_consensus.cc:686] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852 [term 1 LEADER]: Becoming Leader. State: Replica: 74190da3ddc84b1dbeb079b114c55852, State: Running, Role: LEADER
I20260812 07:02:28.488903 14818 consensus_queue.cc:237] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } }
I20260812 07:02:28.493090 13919 catalog_manager.cc:5495] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74190da3ddc84b1dbeb079b114c55852 (127.11.192.194). New cstate: current_term: 1 leader_uuid: "74190da3ddc84b1dbeb079b114c55852" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } health_report { overall_health: UNKNOWN } } }
I20260812 07:02:28.529661 14796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.524335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42174 (local address 127.11.192.194:37823)
0812 07:02:28.524541 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:28.524548 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:28.524665 (+ 117us) server_negotiation.cc:407] Connection header received
0812 07:02:28.524968 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:28.524972 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:28.525060 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:28.525182 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:28.525545 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.526263 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.527563 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.527967 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.528097 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.528579 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:28.529065 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:28.529195 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:28.529325 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 07:02:28.529407 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:02:28.537385 14686 raft_consensus.cc:1261] T c5dafb0ef6394dc1af1845c511be13a6 P 6e2311c83d1044b3afd795ce6e02ce49 [term 1 FOLLOWER]: Refusing update from remote peer 74190da3ddc84b1dbeb079b114c55852: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:28.537381 14203 raft_consensus.cc:1261] T c5dafb0ef6394dc1af1845c511be13a6 P 26f53c403df140bc8ce7e43c59dcb10b [term 1 FOLLOWER]: Refusing update from remote peer 74190da3ddc84b1dbeb079b114c55852: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:28.538371 14826 consensus_queue.cc:1035] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:02:28.538744 14818 consensus_queue.cc:1035] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:02:28.550093 14835 mvcc.cc:205] Tried to move back new op lower bound from 7317578336398905344 to 7317578336214315008. Current Snapshot: MvccSnapshot[applied={T|T < 7317578336398905344}]
I20260812 07:02:28.565376 13919 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:02:28.565662 13919 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:02:28.569367 13919 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b: Sending DeleteTablet for 3 replicas of tablet c5dafb0ef6394dc1af1845c511be13a6
I20260812 07:02:28.570338 14156 tablet_service.cc:1514] Processing DeleteTablet for tablet c5dafb0ef6394dc1af1845c511be13a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:28 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:35854
I20260812 07:02:28.570436 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet c5dafb0ef6394dc1af1845c511be13a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:28 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:42170
I20260812 07:02:28.570470 14645 tablet_service.cc:1514] Processing DeleteTablet for tablet c5dafb0ef6394dc1af1845c511be13a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:28 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:54598
I20260812 07:02:28.574762 13921 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:02:28.575450 13921 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:02:28.583590 14155 tablet_service.cc:1467] Processing CreateTablet for tablet 070143f858404a1b977bc89c8ae68bdc (DEFAULT_TABLE table=test-table [id=d29844748bc644d38160660ff46ed53e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:28.584229 14155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 070143f858404a1b977bc89c8ae68bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:28.587244 14851 tablet_replica.cc:330] T c5dafb0ef6394dc1af1845c511be13a6 P 6e2311c83d1044b3afd795ce6e02ce49: stopping tablet replica
I20260812 07:02:28.587687 14851 raft_consensus.cc:2229] T c5dafb0ef6394dc1af1845c511be13a6 P 6e2311c83d1044b3afd795ce6e02ce49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:02:28.588299 14851 raft_consensus.cc:2258] T c5dafb0ef6394dc1af1845c511be13a6 P 6e2311c83d1044b3afd795ce6e02ce49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:28.589728 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 070143f858404a1b977bc89c8ae68bdc (DEFAULT_TABLE table=test-table [id=d29844748bc644d38160660ff46ed53e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:28.589936 14645 tablet_service.cc:1467] Processing CreateTablet for tablet 070143f858404a1b977bc89c8ae68bdc (DEFAULT_TABLE table=test-table [id=d29844748bc644d38160660ff46ed53e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:28.590322 14645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 070143f858404a1b977bc89c8ae68bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:28.590348 14400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 070143f858404a1b977bc89c8ae68bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:28.595304 14808 tablet_bootstrap.cc:492] T 070143f858404a1b977bc89c8ae68bdc P 26f53c403df140bc8ce7e43c59dcb10b: Bootstrap starting.
I20260812 07:02:28.596889 14808 tablet_bootstrap.cc:654] T 070143f858404a1b977bc89c8ae68bdc P 26f53c403df140bc8ce7e43c59dcb10b: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:28.597548 14806 tablet_bootstrap.cc:492] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852: Bootstrap starting.
I20260812 07:02:28.599182 14806 tablet_bootstrap.cc:654] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:28.600281 14812 tablet_bootstrap.cc:492] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49: Bootstrap starting.
I20260812 07:02:28.601858 14812 tablet_bootstrap.cc:654] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:28.606801 14854 tablet_replica.cc:330] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852: stopping tablet replica
I20260812 07:02:28.607211 14851 ts_tablet_manager.cc:1905] T c5dafb0ef6394dc1af1845c511be13a6 P 6e2311c83d1044b3afd795ce6e02ce49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:28.610153 14808 tablet_bootstrap.cc:492] T 070143f858404a1b977bc89c8ae68bdc P 26f53c403df140bc8ce7e43c59dcb10b: No bootstrap required, opened a new log
I20260812 07:02:28.610365 14808 ts_tablet_manager.cc:1397] T 070143f858404a1b977bc89c8ae68bdc P 26f53c403df140bc8ce7e43c59dcb10b: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.008s
I20260812 07:02:28.611235 14808 raft_consensus.cc:348] T 070143f858404a1b977bc89c8ae68bdc P 26f53c403df140bc8ce7e43c59dcb10b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } }
I20260812 07:02:28.611559 14808 raft_consensus.cc:374] T 070143f858404a1b977bc89c8ae68bdc P 26f53c403df140bc8ce7e43c59dcb10b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:28.611687 14808 raft_consensus.cc:729] T 070143f858404a1b977bc89c8ae68bdc P 26f53c403df140bc8ce7e43c59dcb10b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26f53c403df140bc8ce7e43c59dcb10b, State: Initialized, Role: FOLLOWER
I20260812 07:02:28.612035 14808 consensus_queue.cc:260] T 070143f858404a1b977bc89c8ae68bdc P 26f53c403df140bc8ce7e43c59dcb10b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } }
I20260812 07:02:28.613696 14854 raft_consensus.cc:2229] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:02:28.614110 14812 tablet_bootstrap.cc:492] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49: No bootstrap required, opened a new log
I20260812 07:02:28.613231 14806 tablet_bootstrap.cc:492] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852: No bootstrap required, opened a new log
I20260812 07:02:28.614260 14812 ts_tablet_manager.cc:1397] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.007s
I20260812 07:02:28.614423 14806 ts_tablet_manager.cc:1397] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.001s
I20260812 07:02:28.615103 14808 ts_tablet_manager.cc:1428] T 070143f858404a1b977bc89c8ae68bdc P 26f53c403df140bc8ce7e43c59dcb10b: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260812 07:02:28.615031 14812 raft_consensus.cc:348] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } }
I20260812 07:02:28.615262 14812 raft_consensus.cc:374] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:28.615357 14812 raft_consensus.cc:729] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e2311c83d1044b3afd795ce6e02ce49, State: Initialized, Role: FOLLOWER
I20260812 07:02:28.615674 14812 consensus_queue.cc:260] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } }
I20260812 07:02:28.615751 14806 raft_consensus.cc:348] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } }
I20260812 07:02:28.615973 14806 raft_consensus.cc:374] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:28.616066 14806 raft_consensus.cc:729] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74190da3ddc84b1dbeb079b114c55852, State: Initialized, Role: FOLLOWER
I20260812 07:02:28.616372 14806 consensus_queue.cc:260] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } }
I20260812 07:02:28.617085 14812 ts_tablet_manager.cc:1428] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 07:02:28.617794 14849 tablet_replica.cc:330] T c5dafb0ef6394dc1af1845c511be13a6 P 26f53c403df140bc8ce7e43c59dcb10b: stopping tablet replica
I20260812 07:02:28.618600 14806 ts_tablet_manager.cc:1428] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260812 07:02:28.619024 14849 raft_consensus.cc:2229] T c5dafb0ef6394dc1af1845c511be13a6 P 26f53c403df140bc8ce7e43c59dcb10b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:02:28.619187 14854 raft_consensus.cc:2258] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:28.619608 14849 raft_consensus.cc:2258] T c5dafb0ef6394dc1af1845c511be13a6 P 26f53c403df140bc8ce7e43c59dcb10b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:28.621236 14854 ts_tablet_manager.cc:1905] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:28.621928 14851 ts_tablet_manager.cc:1918] T c5dafb0ef6394dc1af1845c511be13a6 P 6e2311c83d1044b3afd795ce6e02ce49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:02:28.622135 14851 log.cc:1196] T c5dafb0ef6394dc1af1845c511be13a6 P 6e2311c83d1044b3afd795ce6e02ce49: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/c5dafb0ef6394dc1af1845c511be13a6
I20260812 07:02:28.622669 14851 ts_tablet_manager.cc:1939] T c5dafb0ef6394dc1af1845c511be13a6 P 6e2311c83d1044b3afd795ce6e02ce49: Deleting consensus metadata
I20260812 07:02:28.624480 13896 catalog_manager.cc:4841] TS 6e2311c83d1044b3afd795ce6e02ce49 (127.11.192.195:33171): tablet c5dafb0ef6394dc1af1845c511be13a6 (table test-table [id=fa2053ee6f3e4261838dd0ebfdb43e69]) successfully deleted
I20260812 07:02:28.624815 14849 ts_tablet_manager.cc:1905] T c5dafb0ef6394dc1af1845c511be13a6 P 26f53c403df140bc8ce7e43c59dcb10b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:28.628051 14854 ts_tablet_manager.cc:1918] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:02:28.628261 14854 log.cc:1196] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/c5dafb0ef6394dc1af1845c511be13a6
I20260812 07:02:28.628715 14854 ts_tablet_manager.cc:1939] T c5dafb0ef6394dc1af1845c511be13a6 P 74190da3ddc84b1dbeb079b114c55852: Deleting consensus metadata
I20260812 07:02:28.630127 13899 catalog_manager.cc:4841] TS 74190da3ddc84b1dbeb079b114c55852 (127.11.192.194:37823): tablet c5dafb0ef6394dc1af1845c511be13a6 (table test-table [id=fa2053ee6f3e4261838dd0ebfdb43e69]) successfully deleted
I20260812 07:02:28.632061 14849 ts_tablet_manager.cc:1918] T c5dafb0ef6394dc1af1845c511be13a6 P 26f53c403df140bc8ce7e43c59dcb10b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:02:28.632288 14849 log.cc:1196] T c5dafb0ef6394dc1af1845c511be13a6 P 26f53c403df140bc8ce7e43c59dcb10b: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-0/wal/wals/c5dafb0ef6394dc1af1845c511be13a6
I20260812 07:02:28.632809 14849 ts_tablet_manager.cc:1939] T c5dafb0ef6394dc1af1845c511be13a6 P 26f53c403df140bc8ce7e43c59dcb10b: Deleting consensus metadata
I20260812 07:02:28.634527 13898 catalog_manager.cc:4841] TS 26f53c403df140bc8ce7e43c59dcb10b (127.11.192.193:38347): tablet c5dafb0ef6394dc1af1845c511be13a6 (table test-table [id=fa2053ee6f3e4261838dd0ebfdb43e69]) successfully deleted
I20260812 07:02:28.728325 14822 raft_consensus.cc:482] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:02:28.728574 14822 raft_consensus.cc:504] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } }
I20260812 07:02:28.729803 14826 raft_consensus.cc:482] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:02:28.730027 14826 raft_consensus.cc:504] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } }
I20260812 07:02:28.730491 14822 leader_election.cc:288] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 26f53c403df140bc8ce7e43c59dcb10b (127.11.192.193:38347), 74190da3ddc84b1dbeb079b114c55852 (127.11.192.194:37823)
I20260812 07:02:28.730722 14826 leader_election.cc:288] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 26f53c403df140bc8ce7e43c59dcb10b (127.11.192.193:38347), 6e2311c83d1044b3afd795ce6e02ce49 (127.11.192.195:33171)
I20260812 07:02:28.731845 14203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "070143f858404a1b977bc89c8ae68bdc" candidate_uuid: "74190da3ddc84b1dbeb079b114c55852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26f53c403df140bc8ce7e43c59dcb10b" is_pre_election: true
I20260812 07:02:28.732163 14203 raft_consensus.cc:2454] T 070143f858404a1b977bc89c8ae68bdc P 26f53c403df140bc8ce7e43c59dcb10b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74190da3ddc84b1dbeb079b114c55852 in term 0.
I20260812 07:02:28.732149 14686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "070143f858404a1b977bc89c8ae68bdc" candidate_uuid: "74190da3ddc84b1dbeb079b114c55852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2311c83d1044b3afd795ce6e02ce49" is_pre_election: true
I20260812 07:02:28.732357 14686 raft_consensus.cc:2454] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74190da3ddc84b1dbeb079b114c55852 in term 0.
I20260812 07:02:28.732627 14318 leader_election.cc:302] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26f53c403df140bc8ce7e43c59dcb10b, 74190da3ddc84b1dbeb079b114c55852; no voters:
I20260812 07:02:28.732887 14826 raft_consensus.cc:2781] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:02:28.732986 14826 raft_consensus.cc:482] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:02:28.733115 14826 raft_consensus.cc:3037] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:28.736124 14826 raft_consensus.cc:504] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } }
I20260812 07:02:28.737226 14686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "070143f858404a1b977bc89c8ae68bdc" candidate_uuid: "74190da3ddc84b1dbeb079b114c55852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2311c83d1044b3afd795ce6e02ce49"
I20260812 07:02:28.737411 14686 raft_consensus.cc:3037] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:28.740427 14686 raft_consensus.cc:2454] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74190da3ddc84b1dbeb079b114c55852 in term 1.
I20260812 07:02:28.740852 14317 leader_election.cc:302] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2311c83d1044b3afd795ce6e02ce49, 74190da3ddc84b1dbeb079b114c55852; no voters:
I20260812 07:02:28.741137 14818 raft_consensus.cc:2781] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:02:28.741168 14826 leader_election.cc:288] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852 [CANDIDATE]: Term 1 election: Requested vote from peers 26f53c403df140bc8ce7e43c59dcb10b (127.11.192.193:38347), 6e2311c83d1044b3afd795ce6e02ce49 (127.11.192.195:33171)
I20260812 07:02:28.742121 14818 raft_consensus.cc:686] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852 [term 1 LEADER]: Becoming Leader. State: Replica: 74190da3ddc84b1dbeb079b114c55852, State: Running, Role: LEADER
I20260812 07:02:28.742130 14203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "070143f858404a1b977bc89c8ae68bdc" candidate_uuid: "74190da3ddc84b1dbeb079b114c55852" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26f53c403df140bc8ce7e43c59dcb10b"
I20260812 07:02:28.742341 14203 raft_consensus.cc:3037] T 070143f858404a1b977bc89c8ae68bdc P 26f53c403df140bc8ce7e43c59dcb10b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:28.742446 14818 consensus_queue.cc:237] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } }
I20260812 07:02:28.745394 14203 raft_consensus.cc:2454] T 070143f858404a1b977bc89c8ae68bdc P 26f53c403df140bc8ce7e43c59dcb10b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74190da3ddc84b1dbeb079b114c55852 in term 1.
I20260812 07:02:28.746524 14574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.731097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:38347 (local address 127.11.192.195:47049)
0812 07:02:28.731473 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:28.731492 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:28.731619 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:28.732549 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:28.732554 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:28.732581 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:28.732773 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:28.732784 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.734226 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.734231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:28.736247 (+ 2016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:28.736263 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.745719 (+ 9456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.745725 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:28.745966 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.746002 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 07:02:28.746097 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 07:02:28.746238 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255}
I20260812 07:02:28.747195 14795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.731907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:47049 (local address 127.11.192.193:38347)
0812 07:02:28.732103 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:28.732109 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:28.732131 (+ 22us) server_negotiation.cc:407] Connection header received
0812 07:02:28.732201 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:28.732207 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:28.732278 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:28.732360 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:28.732938 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.734087 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.743550 (+ 9463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.745069 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.745553 (+ 484us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.745630 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:28.746795 (+ 1165us) server_negotiation.cc:299] Negotiation successful
0812 07:02:28.746944 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:02:28.747711 14203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "070143f858404a1b977bc89c8ae68bdc" candidate_uuid: "6e2311c83d1044b3afd795ce6e02ce49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26f53c403df140bc8ce7e43c59dcb10b" is_pre_election: true
I20260812 07:02:28.748004 14203 raft_consensus.cc:2379] T 070143f858404a1b977bc89c8ae68bdc P 26f53c403df140bc8ce7e43c59dcb10b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e2311c83d1044b3afd795ce6e02ce49 in current term 1: Already voted for candidate 74190da3ddc84b1dbeb079b114c55852 in this term.
I20260812 07:02:28.749517 14796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.731147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:34901 (local address 127.11.192.194:37823)
0812 07:02:28.731324 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:28.731330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:28.741851 (+ 10521us) server_negotiation.cc:407] Connection header received
0812 07:02:28.742176 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:28.742181 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:28.742309 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:28.742378 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:28.743281 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.744406 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.747151 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.748354 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.748461 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.748545 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:28.749096 (+ 551us) server_negotiation.cc:299] Negotiation successful
0812 07:02:28.749214 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:02:28.750211 14862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.731103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:37823 (local address 127.11.192.195:34901)
0812 07:02:28.741754 (+ 10651us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:28.741773 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:28.741961 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:28.742658 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:28.742668 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:28.742696 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:28.743037 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:28.743049 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.744629 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.744635 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:28.746958 (+ 2323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:28.746975 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.748747 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.748754 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:28.748911 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.748943 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 07:02:28.749727 (+ 784us) client_negotiation.cc:240] Negotiation successful
0812 07:02:28.749855 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10497,"mutex_wait_us":3094,"thread_start_us":7327,"threads_started":1}
I20260812 07:02:28.749922 13921 catalog_manager.cc:5495] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74190da3ddc84b1dbeb079b114c55852 (127.11.192.194). New cstate: current_term: 1 leader_uuid: "74190da3ddc84b1dbeb079b114c55852" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } health_report { overall_health: UNKNOWN } } }
I20260812 07:02:28.751042 14437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "070143f858404a1b977bc89c8ae68bdc" candidate_uuid: "6e2311c83d1044b3afd795ce6e02ce49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74190da3ddc84b1dbeb079b114c55852" is_pre_election: true
I20260812 07:02:28.752171 14568 leader_election.cc:302] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e2311c83d1044b3afd795ce6e02ce49; no voters: 26f53c403df140bc8ce7e43c59dcb10b, 74190da3ddc84b1dbeb079b114c55852
I20260812 07:02:28.752552 14822 raft_consensus.cc:2726] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260812 07:02:28.803452 14771 tablet.cc:2369] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:02:28.804227 14527 tablet.cc:2369] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:02:28.818979 14289 tablet.cc:2369] T 070143f858404a1b977bc89c8ae68bdc P 26f53c403df140bc8ce7e43c59dcb10b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 07:02:28 dist-test-slave-lgr5 krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518146, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:28.834003 14878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.818175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43965 (local address 127.0.0.1:40720)
0812 07:02:28.818818 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:28.818836 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:28.819018 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:28.819476 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:28.819481 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:28.820358 (+ 877us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:28.820691 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:28.820706 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.822155 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.822161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:28.823713 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:28.823729 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.824812 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.824819 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:28.825008 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.825753 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:28.825775 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:28.828034 (+ 2259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:28.831620 (+ 3586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:28.831630 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:28.831635 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:28.832058 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:28.832503 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:28.832506 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:28.832509 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:28.832636 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:28.833222 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:28.833227 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:28.833367 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 07:02:28.833505 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 07:02:28.833696 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":340,"threads_started":1}
I20260812 07:02:28.834263 14002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.818123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40720 (local address 127.11.192.254:43965)
0812 07:02:28.818310 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:28.818316 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:28.818910 (+ 594us) server_negotiation.cc:407] Connection header received
0812 07:02:28.819147 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:28.819153 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:28.819225 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:28.819293 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:28.820865 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.822005 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.823871 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.824621 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.824758 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.828236 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:28.828243 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:28.828248 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:28.828286 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:28.831488 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:28.832215 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:28.832220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:28.832224 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:28.832389 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:28.832745 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:28.832750 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:28.832753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:28.833102 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:28.833248 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:28.833649 (+ 401us) server_negotiation.cc:299] Negotiation successful
0812 07:02:28.833875 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:02:28.839344 13921 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:40720:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 07:02:28.840543 13921 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:02:28.847621 14645 tablet_service.cc:1467] Processing CreateTablet for tablet 361bc1c89bcf4920a36e11bce9deec2e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=78e83a5d4301416d82abbaf9d4a857e6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:02:28.847625 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 361bc1c89bcf4920a36e11bce9deec2e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=78e83a5d4301416d82abbaf9d4a857e6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:02:28.848235 14401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 361bc1c89bcf4920a36e11bce9deec2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:28.848232 14645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 361bc1c89bcf4920a36e11bce9deec2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:28.847919 14156 tablet_service.cc:1467] Processing CreateTablet for tablet 361bc1c89bcf4920a36e11bce9deec2e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=78e83a5d4301416d82abbaf9d4a857e6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:02:28.848610 14156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 361bc1c89bcf4920a36e11bce9deec2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:28.855459 14812 tablet_bootstrap.cc:492] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49: Bootstrap starting.
I20260812 07:02:28.855615 14808 tablet_bootstrap.cc:492] T 361bc1c89bcf4920a36e11bce9deec2e P 26f53c403df140bc8ce7e43c59dcb10b: Bootstrap starting.
I20260812 07:02:28.857197 14812 tablet_bootstrap.cc:654] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:28.857197 14808 tablet_bootstrap.cc:654] T 361bc1c89bcf4920a36e11bce9deec2e P 26f53c403df140bc8ce7e43c59dcb10b: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:28.858124 14806 tablet_bootstrap.cc:492] T 361bc1c89bcf4920a36e11bce9deec2e P 74190da3ddc84b1dbeb079b114c55852: Bootstrap starting.
I20260812 07:02:28.859910 14806 tablet_bootstrap.cc:654] T 361bc1c89bcf4920a36e11bce9deec2e P 74190da3ddc84b1dbeb079b114c55852: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:28.869611 14806 tablet_bootstrap.cc:492] T 361bc1c89bcf4920a36e11bce9deec2e P 74190da3ddc84b1dbeb079b114c55852: No bootstrap required, opened a new log
I20260812 07:02:28.869843 14806 ts_tablet_manager.cc:1397] T 361bc1c89bcf4920a36e11bce9deec2e P 74190da3ddc84b1dbeb079b114c55852: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.004s
I20260812 07:02:28.870687 14806 raft_consensus.cc:348] T 361bc1c89bcf4920a36e11bce9deec2e P 74190da3ddc84b1dbeb079b114c55852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } } peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } }
I20260812 07:02:28.871143 14806 raft_consensus.cc:374] T 361bc1c89bcf4920a36e11bce9deec2e P 74190da3ddc84b1dbeb079b114c55852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:28.871255 14806 raft_consensus.cc:729] T 361bc1c89bcf4920a36e11bce9deec2e P 74190da3ddc84b1dbeb079b114c55852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74190da3ddc84b1dbeb079b114c55852, State: Initialized, Role: FOLLOWER
I20260812 07:02:28.871558 14806 consensus_queue.cc:260] T 361bc1c89bcf4920a36e11bce9deec2e P 74190da3ddc84b1dbeb079b114c55852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } } peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } }
I20260812 07:02:28.872308 14812 tablet_bootstrap.cc:492] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49: No bootstrap required, opened a new log
I20260812 07:02:28.872438 14808 tablet_bootstrap.cc:492] T 361bc1c89bcf4920a36e11bce9deec2e P 26f53c403df140bc8ce7e43c59dcb10b: No bootstrap required, opened a new log
I20260812 07:02:28.872479 14812 ts_tablet_manager.cc:1397] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.001s
I20260812 07:02:28.872679 14808 ts_tablet_manager.cc:1397] T 361bc1c89bcf4920a36e11bce9deec2e P 26f53c403df140bc8ce7e43c59dcb10b: Time spent bootstrapping tablet: real 0.017s user 0.013s sys 0.003s
I20260812 07:02:28.872066 14818 tablet_replica.cc:439] T 361bc1c89bcf4920a36e11bce9deec2e P 74190da3ddc84b1dbeb079b114c55852: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } } peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } }
I20260812 07:02:28.873350 14818 tablet_replica.cc:442] T 361bc1c89bcf4920a36e11bce9deec2e P 74190da3ddc84b1dbeb079b114c55852: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:02:28.873215 14812 raft_consensus.cc:348] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } } peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } }
I20260812 07:02:28.873453 14812 raft_consensus.cc:374] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:28.873550 14812 raft_consensus.cc:729] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e2311c83d1044b3afd795ce6e02ce49, State: Initialized, Role: FOLLOWER
I20260812 07:02:28.873857 14812 consensus_queue.cc:260] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } } peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } }
I20260812 07:02:28.874310 14822 tablet_replica.cc:439] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } } peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } }
I20260812 07:02:28.874575 14822 tablet_replica.cc:442] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:02:28.874388 14808 raft_consensus.cc:348] T 361bc1c89bcf4920a36e11bce9deec2e P 26f53c403df140bc8ce7e43c59dcb10b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } } peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } }
I20260812 07:02:28.874933 14808 raft_consensus.cc:374] T 361bc1c89bcf4920a36e11bce9deec2e P 26f53c403df140bc8ce7e43c59dcb10b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:28.875126 14808 raft_consensus.cc:729] T 361bc1c89bcf4920a36e11bce9deec2e P 26f53c403df140bc8ce7e43c59dcb10b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26f53c403df140bc8ce7e43c59dcb10b, State: Initialized, Role: FOLLOWER
I20260812 07:02:28.875448 14806 ts_tablet_manager.cc:1428] T 361bc1c89bcf4920a36e11bce9deec2e P 74190da3ddc84b1dbeb079b114c55852: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260812 07:02:28.875461 14808 consensus_queue.cc:260] T 361bc1c89bcf4920a36e11bce9deec2e P 26f53c403df140bc8ce7e43c59dcb10b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } } peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } }
I20260812 07:02:28.876266 14808 ts_tablet_manager.cc:1428] T 361bc1c89bcf4920a36e11bce9deec2e P 26f53c403df140bc8ce7e43c59dcb10b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 07:02:28.876731 14821 tablet_replica.cc:439] T 361bc1c89bcf4920a36e11bce9deec2e P 26f53c403df140bc8ce7e43c59dcb10b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } } peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } }
I20260812 07:02:28.877061 14821 tablet_replica.cc:442] T 361bc1c89bcf4920a36e11bce9deec2e P 26f53c403df140bc8ce7e43c59dcb10b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:02:28.878007 14812 ts_tablet_manager.cc:1428] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260812 07:02:28.957260 14822 raft_consensus.cc:482] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:02:28.957445 14822 raft_consensus.cc:504] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } } peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } }
I20260812 07:02:28.958204 14822 leader_election.cc:288] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74190da3ddc84b1dbeb079b114c55852 (127.11.192.194:37823), 26f53c403df140bc8ce7e43c59dcb10b (127.11.192.193:38347)
I20260812 07:02:28.958743 14437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "361bc1c89bcf4920a36e11bce9deec2e" candidate_uuid: "6e2311c83d1044b3afd795ce6e02ce49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74190da3ddc84b1dbeb079b114c55852" is_pre_election: true
I20260812 07:02:28.958932 14203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "361bc1c89bcf4920a36e11bce9deec2e" candidate_uuid: "6e2311c83d1044b3afd795ce6e02ce49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26f53c403df140bc8ce7e43c59dcb10b" is_pre_election: true
I20260812 07:02:28.959077 14437 raft_consensus.cc:2454] T 361bc1c89bcf4920a36e11bce9deec2e P 74190da3ddc84b1dbeb079b114c55852 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e2311c83d1044b3afd795ce6e02ce49 in term 0.
I20260812 07:02:28.959246 14203 raft_consensus.cc:2454] T 361bc1c89bcf4920a36e11bce9deec2e P 26f53c403df140bc8ce7e43c59dcb10b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e2311c83d1044b3afd795ce6e02ce49 in term 0.
I20260812 07:02:28.959535 14568 leader_election.cc:302] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2311c83d1044b3afd795ce6e02ce49, 74190da3ddc84b1dbeb079b114c55852; no voters:
I20260812 07:02:28.959815 14822 raft_consensus.cc:2781] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:02:28.959929 14822 raft_consensus.cc:482] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:02:28.960012 14822 raft_consensus.cc:3037] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:28.962128 14822 raft_consensus.cc:504] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } } peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } }
I20260812 07:02:28.962715 14822 leader_election.cc:288] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49 [CANDIDATE]: Term 1 election: Requested vote from peers 74190da3ddc84b1dbeb079b114c55852 (127.11.192.194:37823), 26f53c403df140bc8ce7e43c59dcb10b (127.11.192.193:38347)
I20260812 07:02:28.963162 14437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "361bc1c89bcf4920a36e11bce9deec2e" candidate_uuid: "6e2311c83d1044b3afd795ce6e02ce49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74190da3ddc84b1dbeb079b114c55852"
I20260812 07:02:28.963192 14203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "361bc1c89bcf4920a36e11bce9deec2e" candidate_uuid: "6e2311c83d1044b3afd795ce6e02ce49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26f53c403df140bc8ce7e43c59dcb10b"
I20260812 07:02:28.963367 14437 raft_consensus.cc:3037] T 361bc1c89bcf4920a36e11bce9deec2e P 74190da3ddc84b1dbeb079b114c55852 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:28.963418 14203 raft_consensus.cc:3037] T 361bc1c89bcf4920a36e11bce9deec2e P 26f53c403df140bc8ce7e43c59dcb10b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:28.965665 14437 raft_consensus.cc:2454] T 361bc1c89bcf4920a36e11bce9deec2e P 74190da3ddc84b1dbeb079b114c55852 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e2311c83d1044b3afd795ce6e02ce49 in term 1.
I20260812 07:02:28.966023 14568 leader_election.cc:302] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2311c83d1044b3afd795ce6e02ce49, 74190da3ddc84b1dbeb079b114c55852; no voters:
I20260812 07:02:28.966198 14203 raft_consensus.cc:2454] T 361bc1c89bcf4920a36e11bce9deec2e P 26f53c403df140bc8ce7e43c59dcb10b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e2311c83d1044b3afd795ce6e02ce49 in term 1.
I20260812 07:02:28.966291 14822 raft_consensus.cc:2781] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:02:28.968315 14822 raft_consensus.cc:686] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49 [term 1 LEADER]: Becoming Leader. State: Replica: 6e2311c83d1044b3afd795ce6e02ce49, State: Running, Role: LEADER
I20260812 07:02:28.968700 14822 consensus_queue.cc:237] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } } peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } }
I20260812 07:02:28.970983 14884 tablet_replica.cc:439] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49: TxnStatusTablet state changed. Reason: New leader 6e2311c83d1044b3afd795ce6e02ce49. Latest consensus state: current_term: 1 leader_uuid: "6e2311c83d1044b3afd795ce6e02ce49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } } peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } }
I20260812 07:02:28.971288 14884 tablet_replica.cc:442] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:02:28.972040 14888 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 07:02:28.972349 14888 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 07:02:28.972302 13921 catalog_manager.cc:5495] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e2311c83d1044b3afd795ce6e02ce49 (127.11.192.195). New cstate: current_term: 1 leader_uuid: "6e2311c83d1044b3afd795ce6e02ce49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } health_report { overall_health: UNKNOWN } } }
I20260812 07:02:28.999203 14892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.991380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54606 (local address 127.11.192.195:33171)
0812 07:02:28.991860 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:28.991866 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:28.992042 (+ 176us) server_negotiation.cc:407] Connection header received
0812 07:02:28.992389 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:28.992395 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:28.992501 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:28.992722 (+ 221us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:28.993161 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.994474 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.996425 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.997132 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.997247 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.997783 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:28.998485 (+ 702us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:28.998622 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:28.998791 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 07:02:28.998875 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":234,"threads_started":1}
I20260812 07:02:28.999193 14878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:28.991741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:33171 (local address 127.0.0.1:54606)
0812 07:02:28.991953 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:28.991972 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:28.992176 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:28.992756 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:28.992762 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:28.992776 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:02:28.992976 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:28.992987 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.994626 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.994632 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:28.996284 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:28.996301 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:28.997300 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:28.997307 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:28.997502 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:28.997588 (+ 86us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:02:28.998613 (+ 1025us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:02:28.998636 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 07:02:28.998728 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 07:02:28.998892 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 07:02:29.004395 14203 raft_consensus.cc:1261] T 361bc1c89bcf4920a36e11bce9deec2e P 26f53c403df140bc8ce7e43c59dcb10b [term 1 FOLLOWER]: Refusing update from remote peer 6e2311c83d1044b3afd795ce6e02ce49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:29.004809 14437 raft_consensus.cc:1261] T 361bc1c89bcf4920a36e11bce9deec2e P 74190da3ddc84b1dbeb079b114c55852 [term 1 FOLLOWER]: Refusing update from remote peer 6e2311c83d1044b3afd795ce6e02ce49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:29.005566 14884 consensus_queue.cc:1035] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:02:29.005895 14822 consensus_queue.cc:1035] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:02:29.021183 14821 tablet_replica.cc:439] T 361bc1c89bcf4920a36e11bce9deec2e P 26f53c403df140bc8ce7e43c59dcb10b: TxnStatusTablet state changed. Reason: New leader 6e2311c83d1044b3afd795ce6e02ce49. Latest consensus state: current_term: 1 leader_uuid: "6e2311c83d1044b3afd795ce6e02ce49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } } peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } }
I20260812 07:02:29.021268 14818 tablet_replica.cc:439] T 361bc1c89bcf4920a36e11bce9deec2e P 74190da3ddc84b1dbeb079b114c55852: TxnStatusTablet state changed. Reason: New leader 6e2311c83d1044b3afd795ce6e02ce49. Latest consensus state: current_term: 1 leader_uuid: "6e2311c83d1044b3afd795ce6e02ce49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } } peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } }
I20260812 07:02:29.021481 14821 tablet_replica.cc:442] T 361bc1c89bcf4920a36e11bce9deec2e P 26f53c403df140bc8ce7e43c59dcb10b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:02:29.021521 14818 tablet_replica.cc:442] T 361bc1c89bcf4920a36e11bce9deec2e P 74190da3ddc84b1dbeb079b114c55852: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:02:29.021557 14884 tablet_replica.cc:439] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6e2311c83d1044b3afd795ce6e02ce49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } } peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } }
I20260812 07:02:29.021811 14884 tablet_replica.cc:442] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:02:29.025036 14834 mvcc.cc:205] Tried to move back new op lower bound from 7317578338313695232 to 7317578338179407872. Current Snapshot: MvccSnapshot[applied={T|T < 7317578338313695232}]
I20260812 07:02:29.026010 14821 tablet_replica.cc:439] T 361bc1c89bcf4920a36e11bce9deec2e P 26f53c403df140bc8ce7e43c59dcb10b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6e2311c83d1044b3afd795ce6e02ce49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } } peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } }
I20260812 07:02:29.026232 14821 tablet_replica.cc:442] T 361bc1c89bcf4920a36e11bce9deec2e P 26f53c403df140bc8ce7e43c59dcb10b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:02:29.026327 14822 tablet_replica.cc:439] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6e2311c83d1044b3afd795ce6e02ce49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } } peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } }
I20260812 07:02:29.026583 14822 tablet_replica.cc:442] T 361bc1c89bcf4920a36e11bce9deec2e P 6e2311c83d1044b3afd795ce6e02ce49: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:02:29.029032 14818 tablet_replica.cc:439] T 361bc1c89bcf4920a36e11bce9deec2e P 74190da3ddc84b1dbeb079b114c55852: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6e2311c83d1044b3afd795ce6e02ce49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74190da3ddc84b1dbeb079b114c55852" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 37823 } } peers { permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 } } peers { permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 } } }
I20260812 07:02:29.029309 14818 tablet_replica.cc:442] T 361bc1c89bcf4920a36e11bce9deec2e P 74190da3ddc84b1dbeb079b114c55852: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:02:29.046113 14899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:29.040566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:33171 (local address 127.11.192.194:48603)
0812 07:02:29.041078 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:29.041093 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:29.041227 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:29.041526 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:29.041531 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:29.041543 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:02:29.041776 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:29.041787 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:29.042919 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:29.042924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:29.044232 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:29.044247 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:29.044962 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:29.044968 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:29.045149 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:29.045207 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:02:29.045642 (+ 435us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:02:29.045649 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 07:02:29.045705 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 07:02:29.045802 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":239,"threads_started":1}
I20260812 07:02:29.046123 14892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:29.040599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:48603 (local address 127.11.192.195:33171)
0812 07:02:29.040755 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:29.040762 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:29.041175 (+ 413us) server_negotiation.cc:407] Connection header received
0812 07:02:29.041360 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:29.041364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:29.041425 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:29.041495 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:29.041927 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:29.042799 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:29.044371 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:29.044806 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:29.044952 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:29.045405 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:29.045550 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:29.045645 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:29.045741 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 07:02:29.045830 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 07:02:29.055560 14686 raft_consensus.cc:1261] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49 [term 1 FOLLOWER]: Refusing update from remote peer 74190da3ddc84b1dbeb079b114c55852: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:29.055859 14203 raft_consensus.cc:1261] T 070143f858404a1b977bc89c8ae68bdc P 26f53c403df140bc8ce7e43c59dcb10b [term 1 FOLLOWER]: Refusing update from remote peer 74190da3ddc84b1dbeb079b114c55852: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:29.056201 14818 consensus_queue.cc:1035] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2311c83d1044b3afd795ce6e02ce49" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:02:29.056509 14865 consensus_queue.cc:1035] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26f53c403df140bc8ce7e43c59dcb10b" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:02:29.060962 14832 mvcc.cc:205] Tried to move back new op lower bound from 7317578338525982720 to 7317578337251516416. Current Snapshot: MvccSnapshot[applied={T|T < 7317578338525982720}]
I20260812 07:02:29.105841 14796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:29.091160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:57315 (local address 127.11.192.194:37823)
0812 07:02:29.091335 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:29.091358 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:29.096892 (+ 5534us) server_negotiation.cc:407] Connection header received
0812 07:02:29.097032 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:29.097037 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:29.097122 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:29.097217 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:29.097899 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:29.099009 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:29.103375 (+ 4366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:29.103921 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:29.104051 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:29.104877 (+ 826us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:29.105016 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:29.105081 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:29.105557 (+ 476us) server_negotiation.cc:299] Negotiation successful
0812 07:02:29.105641 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:02:29.106072 14905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:29.090744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:37823 (local address 127.11.192.195:57315)
0812 07:02:29.096786 (+ 6042us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:29.096803 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:29.096925 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:29.097257 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:29.097262 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:29.097274 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:02:29.097709 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:29.097722 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:29.099171 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:29.099177 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:29.103185 (+ 4008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:29.103200 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:29.104360 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:29.104368 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:29.104593 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:29.104670 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:02:29.105379 (+ 709us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:02:29.105390 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 07:02:29.105504 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 07:02:29.105653 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5863,"thread_start_us":5739,"threads_started":1}
I20260812 07:02:29.142180 13920 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:02:29.142445 13920 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:02:29.145648 13920 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27f099ac496a47ca8f42644fd741339b: Sending DeleteTablet for 3 replicas of tablet 070143f858404a1b977bc89c8ae68bdc
I20260812 07:02:29.146432 14156 tablet_service.cc:1514] Processing DeleteTablet for tablet 070143f858404a1b977bc89c8ae68bdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:29 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:35854
I20260812 07:02:29.146801 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 070143f858404a1b977bc89c8ae68bdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:29 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:42170
I20260812 07:02:29.147027 14909 tablet_replica.cc:330] T 070143f858404a1b977bc89c8ae68bdc P 26f53c403df140bc8ce7e43c59dcb10b: stopping tablet replica
I20260812 07:02:29.147257 14909 raft_consensus.cc:2229] T 070143f858404a1b977bc89c8ae68bdc P 26f53c403df140bc8ce7e43c59dcb10b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:02:29.147275 14645 tablet_service.cc:1514] Processing DeleteTablet for tablet 070143f858404a1b977bc89c8ae68bdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:29 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:54598
I20260812 07:02:29.147513 14909 raft_consensus.cc:2258] T 070143f858404a1b977bc89c8ae68bdc P 26f53c403df140bc8ce7e43c59dcb10b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:29.147465 14912 tablet_replica.cc:330] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852: stopping tablet replica
I20260812 07:02:29.147912 14912 raft_consensus.cc:2229] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:02:29.148476 14912 raft_consensus.cc:2258] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:29.149737 14912 ts_tablet_manager.cc:1905] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:29.154702 14909 ts_tablet_manager.cc:1905] T 070143f858404a1b977bc89c8ae68bdc P 26f53c403df140bc8ce7e43c59dcb10b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:29.155582 14913 tablet_replica.cc:330] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49: stopping tablet replica
I20260812 07:02:29.155788 14913 raft_consensus.cc:2229] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:02:29.156075 14913 raft_consensus.cc:2258] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:29.157196 14913 ts_tablet_manager.cc:1905] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:29.157612 14912 ts_tablet_manager.cc:1918] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:02:29.157788 14912 log.cc:1196] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/070143f858404a1b977bc89c8ae68bdc
I20260812 07:02:29.158289 14912 ts_tablet_manager.cc:1939] T 070143f858404a1b977bc89c8ae68bdc P 74190da3ddc84b1dbeb079b114c55852: Deleting consensus metadata
I20260812 07:02:29.159384 13899 catalog_manager.cc:4841] TS 74190da3ddc84b1dbeb079b114c55852 (127.11.192.194:37823): tablet 070143f858404a1b977bc89c8ae68bdc (table test-table [id=d29844748bc644d38160660ff46ed53e]) successfully deleted
I20260812 07:02:29.162566 14909 ts_tablet_manager.cc:1918] T 070143f858404a1b977bc89c8ae68bdc P 26f53c403df140bc8ce7e43c59dcb10b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:02:29.162719 14909 log.cc:1196] T 070143f858404a1b977bc89c8ae68bdc P 26f53c403df140bc8ce7e43c59dcb10b: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-0/wal/wals/070143f858404a1b977bc89c8ae68bdc
I20260812 07:02:29.163216 14909 ts_tablet_manager.cc:1939] T 070143f858404a1b977bc89c8ae68bdc P 26f53c403df140bc8ce7e43c59dcb10b: Deleting consensus metadata
I20260812 07:02:29.164113 14913 ts_tablet_manager.cc:1918] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:02:29.164252 14913 log.cc:1196] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/070143f858404a1b977bc89c8ae68bdc
I20260812 07:02:29.164399 13898 catalog_manager.cc:4841] TS 26f53c403df140bc8ce7e43c59dcb10b (127.11.192.193:38347): tablet 070143f858404a1b977bc89c8ae68bdc (table test-table [id=d29844748bc644d38160660ff46ed53e]) successfully deleted
I20260812 07:02:29.164749 14913 ts_tablet_manager.cc:1939] T 070143f858404a1b977bc89c8ae68bdc P 6e2311c83d1044b3afd795ce6e02ce49: Deleting consensus metadata
I20260812 07:02:29.165864 13896 catalog_manager.cc:4841] TS 6e2311c83d1044b3afd795ce6e02ce49 (127.11.192.195:33171): tablet 070143f858404a1b977bc89c8ae68bdc (table test-table [id=d29844748bc644d38160660ff46ed53e]) successfully deleted
Aug 12 07:02:29 dist-test-slave-lgr5 krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518148, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.192.193@KRBTEST.COM
I20260812 07:02:29.183058 14795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:29.161183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35858 (local address 127.11.192.193:38347)
0812 07:02:29.161383 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:29.161389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:29.168627 (+ 7238us) server_negotiation.cc:407] Connection header received
0812 07:02:29.168793 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:29.168797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:29.168870 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:29.168953 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:29.170368 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:29.171059 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:29.172333 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:29.172722 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:29.172839 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:29.177288 (+ 4449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:29.177301 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:29.177318 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:29.177350 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:29.180238 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:29.181051 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:29.181058 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:29.181061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:29.181192 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:29.181582 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:29.181585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:29.181587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:29.181939 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:29.182232 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:29.182582 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 07:02:29.182783 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 07:02:29.183959 14274 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35858
I20260812 07:02:29.200172 14002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:29.187094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40730 (local address 127.11.192.254:43965)
0812 07:02:29.187318 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:29.187324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:29.187496 (+ 172us) server_negotiation.cc:407] Connection header received
0812 07:02:29.187693 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:29.187699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:29.187769 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:29.187849 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:29.189669 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:29.190390 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:29.192226 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:29.192669 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:29.192794 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:29.195057 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:29.195062 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:29.195065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:29.195098 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:29.197687 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:29.198309 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:29.198315 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:29.198319 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:29.198447 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:29.198785 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:29.198790 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:29.198792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:29.199057 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:29.199297 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:29.199653 (+ 356us) server_negotiation.cc:299] Negotiation successful
0812 07:02:29.199866 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260812 07:02:29.200758 13920 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40730
I20260812 07:02:29.203153 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 14040
I20260812 07:02:29.215788 14795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:29.203960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35874 (local address 127.11.192.193:38347)
0812 07:02:29.204534 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:29.204541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:29.204940 (+ 399us) server_negotiation.cc:407] Connection header received
0812 07:02:29.205096 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:29.205100 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:29.205161 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:29.205241 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:29.206295 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:29.207061 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:29.208274 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:29.208670 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:29.208779 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:29.210834 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:29.210839 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:29.210842 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:29.210874 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:29.213397 (+ 2523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:29.213995 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:29.214000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:29.214002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:29.214131 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:29.214459 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:29.214464 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:29.214467 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:29.214730 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:29.214983 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:29.215257 (+ 274us) server_negotiation.cc:299] Negotiation successful
0812 07:02:29.215464 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424}
W20260812 07:02:29.216447 14274 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35874
W20260812 07:02:29.216986 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 14040: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:02:29.217190 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 14040
I20260812 07:02:29.250612 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 14293
Aug 12 07:02:29 dist-test-slave-lgr5 krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518148, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.192.194@KRBTEST.COM
I20260812 07:02:29.265460 14796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:29.251382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42186 (local address 127.11.192.194:37823)
0812 07:02:29.251576 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:29.251582 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:29.251627 (+ 45us) server_negotiation.cc:407] Connection header received
0812 07:02:29.251833 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:29.251837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:29.251903 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:29.252002 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:29.253197 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:29.253948 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:29.255182 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:29.255588 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:29.255697 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:29.259845 (+ 4148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:29.259860 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:29.259874 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:29.259909 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:29.262713 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:29.263336 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:29.263344 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:29.263346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:29.263487 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:29.263840 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:29.263844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:29.263847 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:29.264315 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:29.264657 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:29.264923 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 07:02:29.265144 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260812 07:02:29.266103 14515 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42186
W20260812 07:02:29.266724 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 14293: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:02:29.266875 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 14293
I20260812 07:02:29.303718 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 14535
Aug 12 07:02:29 dist-test-slave-lgr5 krb5kdc[13847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518148, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.192.195@KRBTEST.COM
I20260812 07:02:29.324774 14892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:29.305757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54614 (local address 127.11.192.195:33171)
0812 07:02:29.305993 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:29.306000 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:29.306024 (+ 24us) server_negotiation.cc:407] Connection header received
0812 07:02:29.306095 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:29.306102 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:29.306190 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:29.306304 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:29.307991 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:29.309240 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:29.311042 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:29.311709 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:29.311864 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:29.317270 (+ 5406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:29.317284 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:29.317301 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:29.317348 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:29.321404 (+ 4056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:29.322223 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:29.322235 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:29.322239 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:29.322423 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:29.322853 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:29.322858 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:29.322860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:29.323310 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:29.323684 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:29.323918 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 07:02:29.324194 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260812 07:02:29.325487 14760 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54614
W20260812 07:02:29.326215 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 14535: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:02:29.326377 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 14535
I20260812 07:02:29.352993 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 13867
I20260812 07:02:29.354133 13990 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:29.468847 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 13867
2026-08-12T07:02:29Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4565 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:02:29 dist-test-slave-lgr5 krb5kdc[14931](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:02:29 dist-test-slave-lgr5 krb5kdc[14931](info): set up 2 sockets
Aug 12 07:02:29 dist-test-slave-lgr5 krb5kdc[14931](info): commencing operation
krb5kdc: starting...
W20260812 07:02:30.695959 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:02:30 dist-test-slave-lgr5 krb5kdc[14931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:02:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:02:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:02:30.873494 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:46435
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:46435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:31.180616 14955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:31.180877 14955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:31.181008 14955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:31.187978 14955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:02:31.188072 14955 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 07:02:31.188155 14955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:31.188241 14955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:31.188303 14955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:02:31.188388 14955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:02:31.197957 14955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:46435
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:46435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:31.199143 14955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:31.200399 14955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:31.211354 14963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:31.211330 14965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:31.211659 14967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:31.212373 14955 server_base.cc:1034] running on GCE node
I20260812 07:02:31.213054 14955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:31.214351 14955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:31.215566 14955 hybrid_clock.cc:648] HybridClock initialized: now 1786518151215504 us; error 37 us; skew 500 ppm
I20260812 07:02:31.215876 14955 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:31 dist-test-slave-lgr5 krb5kdc[14931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518151, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:31.218832 14955 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:02:31.220156 14955 webserver.cc:460] Webserver started at http://127.11.192.254:39039/ using document root <none> and password file <none>
I20260812 07:02:31.220794 14955 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:31.220886 14955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:31.221180 14955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:31.223981 14955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "ce5f6fcb794a45798500c9063f62535a"
format_stamp: "Formatted at 2026-08-12 07:02:31 on dist-test-slave-lgr5"
server_key: "752375eeff2593fd898197b8a32dd25d"
server_key_iv: "80381f281565d19f11bb522e7f039f74"
server_key_version: "encryptionkey@0"
I20260812 07:02:31.224756 14955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "ce5f6fcb794a45798500c9063f62535a"
format_stamp: "Formatted at 2026-08-12 07:02:31 on dist-test-slave-lgr5"
server_key: "752375eeff2593fd898197b8a32dd25d"
server_key_iv: "80381f281565d19f11bb522e7f039f74"
server_key_version: "encryptionkey@0"
I20260812 07:02:31.229466 14955 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:02:31.232585 14976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:31.233753 14955 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:02:31.233908 14955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "ce5f6fcb794a45798500c9063f62535a"
format_stamp: "Formatted at 2026-08-12 07:02:31 on dist-test-slave-lgr5"
server_key: "752375eeff2593fd898197b8a32dd25d"
server_key_iv: "80381f281565d19f11bb522e7f039f74"
server_key_version: "encryptionkey@0"
I20260812 07:02:31.234104 14955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:31.264406 14955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:31.268653 14955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:31.268886 14955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:31.288367 14955 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:46435
I20260812 07:02:31.288407 15087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:46435 every 8 connection(s)
I20260812 07:02:31.289667 14955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:02:31.293282 15088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:31.298959 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 14955
I20260812 07:02:31.299132 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:02:31.299741 12035 external_mini_cluster.cc:1428] Setting key 5f095fc4d50fb9d7a3abbd928907f877
I20260812 07:02:31.303391 15088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a: Bootstrap starting.
I20260812 07:02:31.306815 15088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:31.307790 15088 log.cc:824] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:31.310603 15088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a: No bootstrap required, opened a new log
Aug 12 07:02:31 dist-test-slave-lgr5 krb5kdc[14931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:31.319455 15088 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5f6fcb794a45798500c9063f62535a" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46435 } }
I20260812 07:02:31.319833 15088 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:31.319939 15088 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce5f6fcb794a45798500c9063f62535a, State: Initialized, Role: FOLLOWER
I20260812 07:02:31.320636 15088 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5f6fcb794a45798500c9063f62535a" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46435 } }
I20260812 07:02:31.320865 15088 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:02:31.321002 15088 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:02:31.321157 15088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:31.321812 15096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:31.301621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45876 (local address 127.11.192.254:46435)
0812 07:02:31.302369 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:31.302379 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:31.302418 (+ 39us) server_negotiation.cc:407] Connection header received
0812 07:02:31.304043 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:31.304064 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:31.304442 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:31.304757 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:31.306478 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:31.307755 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:31.309386 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:31.310022 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:31.310145 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:31.314760 (+ 4615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:31.314768 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:31.314781 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:31.314814 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:31.317802 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:31.318592 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:31.318601 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:31.318608 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:31.318762 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:31.319113 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:31.319116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:31.319118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:31.319601 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:31.319857 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:31.320347 (+ 490us) server_negotiation.cc:299] Negotiation successful
0812 07:02:31.320728 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":354,"threads_started":1}
I20260812 07:02:31.323405 15088 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5f6fcb794a45798500c9063f62535a" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46435 } }
I20260812 07:02:31.323864 15088 leader_election.cc:302] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce5f6fcb794a45798500c9063f62535a; no voters:
I20260812 07:02:31.324478 15088 leader_election.cc:288] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:02:31.324559 15099 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:02:31.325193 15099 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a [term 1 LEADER]: Becoming Leader. State: Replica: ce5f6fcb794a45798500c9063f62535a, State: Running, Role: LEADER
I20260812 07:02:31.325733 15099 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5f6fcb794a45798500c9063f62535a" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46435 } }
I20260812 07:02:31.326525 15088 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:02:31.328550 15101 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce5f6fcb794a45798500c9063f62535a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5f6fcb794a45798500c9063f62535a" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46435 } } }
I20260812 07:02:31.328516 15103 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce5f6fcb794a45798500c9063f62535a. Latest consensus state: current_term: 1 leader_uuid: "ce5f6fcb794a45798500c9063f62535a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5f6fcb794a45798500c9063f62535a" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46435 } } }
I20260812 07:02:31.328984 15101 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:31.329056 15103 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:31.330371 15117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:02:31.335732 15117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:02:31.345227 15117 catalog_manager.cc:1347] Generated new cluster ID: f33edcad11f744bdbe4e70bc2e9a5184
I20260812 07:02:31.345333 15117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:02:31.356796 15117 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:02:31.357925 15117 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:02:31.378969 15117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a: Generated new TSK 0
I20260812 07:02:31.379943 15117 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
I20260812 07:02:31.428645 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:46435
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:31.743364 15143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:31.743654 15143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:31.743814 15143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:31.750828 15143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:31.750923 15143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:31.751045 15143 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 07:02:31.751102 15143 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:02:31.751201 15143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:02:31.760645 15143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.192.254:46435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:31.761806 15143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:31.763041 15143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:31.775447 15155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:31.775487 15165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:31.775516 15161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:31.776289 15143 server_base.cc:1034] running on GCE node
I20260812 07:02:31.776813 15143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:31.777630 15143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:31.778841 15143 hybrid_clock.cc:648] HybridClock initialized: now 1786518151778778 us; error 43 us; skew 500 ppm
I20260812 07:02:31.779210 15143 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:31 dist-test-slave-lgr5 krb5kdc[14931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518151, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:31.784935 15143 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:02:31.786247 15143 webserver.cc:460] Webserver started at http://127.11.192.193:42311/ using document root <none> and password file <none>
I20260812 07:02:31.786877 15143 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:31.786999 15143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:31.787304 15143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:31.790352 15143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "80bc5212b150451b93f08a4ff60cacc5"
format_stamp: "Formatted at 2026-08-12 07:02:31 on dist-test-slave-lgr5"
server_key: "a8d0eec57da25fac03a5e242c672fab1"
server_key_iv: "3ef2ae4c653d5ce3ed16f9d44099a925"
server_key_version: "encryptionkey@0"
I20260812 07:02:31.791100 15143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "80bc5212b150451b93f08a4ff60cacc5"
format_stamp: "Formatted at 2026-08-12 07:02:31 on dist-test-slave-lgr5"
server_key: "a8d0eec57da25fac03a5e242c672fab1"
server_key_iv: "3ef2ae4c653d5ce3ed16f9d44099a925"
server_key_version: "encryptionkey@0"
I20260812 07:02:31.795876 15143 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260812 07:02:31.799057 15176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:31.800302 15143 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:02:31.800451 15143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "80bc5212b150451b93f08a4ff60cacc5"
format_stamp: "Formatted at 2026-08-12 07:02:31 on dist-test-slave-lgr5"
server_key: "a8d0eec57da25fac03a5e242c672fab1"
server_key_iv: "3ef2ae4c653d5ce3ed16f9d44099a925"
server_key_version: "encryptionkey@0"
I20260812 07:02:31.800621 15143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:31.819901 15143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:31.823535 15143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:31.823786 15143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:31.826273 15143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:31.826452 15143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:31.826570 15143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:31.826661 15143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:02:31 dist-test-slave-lgr5 krb5kdc[14931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518151, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:31.847533 15191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:31.826873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:37469 (local address 127.11.192.254:46435)
0812 07:02:31.827646 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:31.827651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:31.828503 (+ 852us) server_negotiation.cc:407] Connection header received
0812 07:02:31.830454 (+ 1951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:31.830460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:31.830553 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:31.830668 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:31.832981 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:31.833883 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:31.836758 (+ 2875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:31.837379 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:31.837512 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:31.841309 (+ 3797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:31.841314 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:31.841317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:31.841351 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:31.844286 (+ 2935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:31.845026 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:31.845032 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:31.845036 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:31.845164 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:31.845769 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:31.845775 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:31.845779 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:31.846054 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:31.846216 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:31.847104 (+ 888us) server_negotiation.cc:299] Negotiation successful
0812 07:02:31.847265 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":560,"threads_started":1}
I20260812 07:02:31.849191 15193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:31.827527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:46435 (local address 127.11.192.193:37469)
0812 07:02:31.828355 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:31.828397 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:31.830197 (+ 1800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:31.830953 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:31.830965 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:31.831769 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:31.832775 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:31.832805 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:31.834077 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:31.834083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:31.836614 (+ 2531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:31.836631 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:31.837645 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:31.837654 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:31.837930 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:31.838603 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:31.838632 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:31.841085 (+ 2453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:31.844415 (+ 3330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:31.844428 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:31.844442 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:31.844907 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:31.845290 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:31.845296 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:31.845301 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:31.845624 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:31.846220 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:31.846235 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:31.846737 (+ 502us) client_negotiation.cc:769] Sending connection context
0812 07:02:31.847043 (+ 306us) client_negotiation.cc:240] Negotiation successful
0812 07:02:31.847578 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":356,"threads_started":1}
I20260812 07:02:31.868484 15191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:31.854514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:48763 (local address 127.11.192.254:46435)
0812 07:02:31.854692 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:31.854697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:31.855298 (+ 601us) server_negotiation.cc:407] Connection header received
0812 07:02:31.855457 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:31.855460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:31.855524 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:31.855589 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:31.856925 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:31.857750 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:31.859630 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:31.860347 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:31.860491 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:31.862554 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:31.862560 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:31.862564 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:31.862599 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:31.865740 (+ 3141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:31.866402 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:31.866407 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:31.866410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:31.866577 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:31.866975 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:31.866981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:31.866985 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:31.867333 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:31.867479 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:31.867915 (+ 436us) server_negotiation.cc:299] Negotiation successful
0812 07:02:31.868160 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:02:31.868718 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:31.854505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:46435 (local address 127.11.192.193:48763)
0812 07:02:31.855184 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:31.855210 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:31.855351 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:31.855631 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:31.855636 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:31.856271 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:31.856763 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:31.856783 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:31.857923 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:31.857929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:31.859457 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:31.859480 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:31.860547 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:31.860555 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:31.860748 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:31.861310 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:31.861334 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:31.862316 (+ 982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:31.865861 (+ 3545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:31.865868 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:31.865870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:31.866261 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:31.866710 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:31.866718 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:31.866721 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:31.866865 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:31.867450 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:31.867455 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:31.867663 (+ 208us) client_negotiation.cc:769] Sending connection context
0812 07:02:31.867790 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 07:02:31.868159 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":367,"threads_started":1}
I20260812 07:02:31.874037 15143 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:34835
I20260812 07:02:31.874068 15376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:34835 every 8 connection(s)
I20260812 07:02:31.875175 15143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:02:31.875639 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 15143
I20260812 07:02:31.875857 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:02:31.876668 12035 external_mini_cluster.cc:1428] Setting key 82fac4ef57887586298fc868ec58d09b
I20260812 07:02:31.877219 15377 heartbeater.cc:347] Connected to a master server at 127.11.192.254:46435
I20260812 07:02:31.877517 15377 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:31.878130 15377 heartbeater.cc:511] Master 127.11.192.254:46435 requested a full tablet report, sending...
I20260812 07:02:31.880146 15010 ts_manager.cc:194] Registered new tserver with Master: 80bc5212b150451b93f08a4ff60cacc5 (127.11.192.193:34835)
I20260812 07:02:31.881641 15010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:37469
WARNING: no policy specified for kudu/127.11.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
I20260812 07:02:31.916374 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:46435
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:32.227027 15382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:32.227278 15382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:32.227394 15382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:32.234403 15382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:32.234503 15382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:32.234588 15382 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 07:02:32.234639 15382 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:02:32.234705 15382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:02:32.243973 15382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.192.254:46435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:32.245143 15382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:32.246356 15382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:32.258605 15392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:32.258642 15389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:32.258679 15388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:32.259397 15382 server_base.cc:1034] running on GCE node
I20260812 07:02:32.259965 15382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:32.260852 15382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:32.262048 15382 hybrid_clock.cc:648] HybridClock initialized: now 1786518152261993 us; error 43 us; skew 500 ppm
I20260812 07:02:32.262346 15382 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:32 dist-test-slave-lgr5 krb5kdc[14931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518152, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:32.265270 15382 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:02:32.266664 15382 webserver.cc:460] Webserver started at http://127.11.192.194:33195/ using document root <none> and password file <none>
I20260812 07:02:32.267287 15382 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:32.267413 15382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:32.267709 15382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:32.270618 15382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "3cd38f56505e46dc8b8109aad1486cd4"
format_stamp: "Formatted at 2026-08-12 07:02:32 on dist-test-slave-lgr5"
server_key: "9708735772ecea73bc6ce639db2bb152"
server_key_iv: "ae2a1e2089e23be6e9738a928e993aea"
server_key_version: "encryptionkey@0"
I20260812 07:02:32.271327 15382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "3cd38f56505e46dc8b8109aad1486cd4"
format_stamp: "Formatted at 2026-08-12 07:02:32 on dist-test-slave-lgr5"
server_key: "9708735772ecea73bc6ce639db2bb152"
server_key_iv: "ae2a1e2089e23be6e9738a928e993aea"
server_key_version: "encryptionkey@0"
I20260812 07:02:32.276229 15382 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260812 07:02:32.279565 15406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:32.280854 15382 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:02:32.281013 15382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "3cd38f56505e46dc8b8109aad1486cd4"
format_stamp: "Formatted at 2026-08-12 07:02:32 on dist-test-slave-lgr5"
server_key: "9708735772ecea73bc6ce639db2bb152"
server_key_iv: "ae2a1e2089e23be6e9738a928e993aea"
server_key_version: "encryptionkey@0"
I20260812 07:02:32.281193 15382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:32.296473 15382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:32.300609 15382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:32.300858 15382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:32.303370 15382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:32.303486 15382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:32.303606 15382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:32.303686 15382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:02:32 dist-test-slave-lgr5 krb5kdc[14931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518152, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:32.323992 15191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:32.303776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:40063 (local address 127.11.192.254:46435)
0812 07:02:32.303994 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:32.304002 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:32.305403 (+ 1401us) server_negotiation.cc:407] Connection header received
0812 07:02:32.308125 (+ 2722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:32.308131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:32.308216 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:32.308317 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:32.310406 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.311507 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.313739 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.314174 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.314286 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:32.318509 (+ 4223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:32.318514 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:32.318517 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:32.318549 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:32.321088 (+ 2539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:32.321768 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:32.321772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:32.321775 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:32.321915 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:32.322345 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:32.322350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:32.322352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:32.322632 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:32.322773 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:32.323429 (+ 656us) server_negotiation.cc:299] Negotiation successful
0812 07:02:32.323648 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 07:02:32.325217 15422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:32.304353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:46435 (local address 127.11.192.194:40063)
0812 07:02:32.305228 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:32.305274 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:32.307728 (+ 2454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:32.308652 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:32.308663 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:32.309368 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:32.310156 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:32.310188 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.311691 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.311696 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:32.313585 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:32.313600 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.314415 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.314423 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:32.314653 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:32.315328 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:32.315359 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:32.318285 (+ 2926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:32.321217 (+ 2932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:32.321226 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:32.321241 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:32.321643 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:32.322022 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:32.322028 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:32.322032 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:32.322225 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:32.322768 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:32.322775 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:32.323079 (+ 304us) client_negotiation.cc:769] Sending connection context
0812 07:02:32.323302 (+ 223us) client_negotiation.cc:240] Negotiation successful
0812 07:02:32.323791 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":333,"threads_started":1}
I20260812 07:02:32.343242 15191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:32.330682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:48295 (local address 127.11.192.254:46435)
0812 07:02:32.330851 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:32.330857 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:32.331408 (+ 551us) server_negotiation.cc:407] Connection header received
0812 07:02:32.331550 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:32.331555 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:32.331661 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:32.331748 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:32.333194 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.334132 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.336045 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.336513 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.336625 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:32.338363 (+ 1738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:32.338367 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:32.338371 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:32.338399 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:32.340819 (+ 2420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:32.341421 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:32.341426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:32.341430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:32.341559 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:32.341891 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:32.341896 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:32.341899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:32.342202 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:32.342334 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:32.342725 (+ 391us) server_negotiation.cc:299] Negotiation successful
0812 07:02:32.342927 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 07:02:32.343356 15501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:32.330701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:46435 (local address 127.11.192.194:48295)
0812 07:02:32.331298 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:32.331313 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:32.331442 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:32.331813 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:32.331822 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:32.332502 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:32.333043 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:32.333057 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.334297 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.334303 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:32.335860 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:32.335876 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.336728 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.336739 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:32.336952 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:32.337462 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:32.337479 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:32.338177 (+ 698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:32.340938 (+ 2761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:32.340943 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:32.340948 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:32.341310 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:32.341663 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:32.341668 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:32.341672 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:32.341791 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:32.342319 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:32.342323 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:32.342502 (+ 179us) client_negotiation.cc:769] Sending connection context
0812 07:02:32.342618 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 07:02:32.342937 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":314,"threads_started":1}
I20260812 07:02:32.355913 15382 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:34241
I20260812 07:02:32.355953 15593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:34241 every 8 connection(s)
I20260812 07:02:32.357112 15382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:02:32.359027 15595 heartbeater.cc:347] Connected to a master server at 127.11.192.254:46435
I20260812 07:02:32.359236 15595 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:32.359853 15595 heartbeater.cc:511] Master 127.11.192.254:46435 requested a full tablet report, sending...
I20260812 07:02:32.361225 15010 ts_manager.cc:194] Registered new tserver with Master: 3cd38f56505e46dc8b8109aad1486cd4 (127.11.192.194:34241)
I20260812 07:02:32.361351 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 15382
I20260812 07:02:32.361526 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:02:32.362104 12035 external_mini_cluster.cc:1428] Setting key bd22597d58c6c0599646cc13f1019b78
I20260812 07:02:32.362306 15010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:40063
WARNING: no policy specified for kudu/127.11.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
I20260812 07:02:32.404017 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:46435
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:32.712584 15608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:32.712831 15608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:32.712965 15608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:32.719799 15608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:32.719897 15608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:32.720018 15608 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 07:02:32.720095 15608 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:02:32.720192 15608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:02:32.729488 15608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.192.254:46435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:32.730608 15608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:32.731902 15608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:32.744135 15619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:32.744177 15627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:32.744405 15629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:32.745250 15608 server_base.cc:1034] running on GCE node
I20260812 07:02:32.745826 15608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:32.746690 15608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:32.747884 15608 hybrid_clock.cc:648] HybridClock initialized: now 1786518152747842 us; error 28 us; skew 500 ppm
I20260812 07:02:32.748214 15608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:32 dist-test-slave-lgr5 krb5kdc[14931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518152, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:32.751163 15608 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:02:32.752573 15608 webserver.cc:460] Webserver started at http://127.11.192.195:41643/ using document root <none> and password file <none>
I20260812 07:02:32.753203 15608 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:32.753320 15608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:32.753611 15608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:32.756417 15608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "186f7554d2ce4a9a838664c036252830"
format_stamp: "Formatted at 2026-08-12 07:02:32 on dist-test-slave-lgr5"
server_key: "b292dd27400e02c0bcab9320f1c70ec5"
server_key_iv: "62b4e3217178181bc3cc2b3d72c2f695"
server_key_version: "encryptionkey@0"
I20260812 07:02:32.757135 15608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "186f7554d2ce4a9a838664c036252830"
format_stamp: "Formatted at 2026-08-12 07:02:32 on dist-test-slave-lgr5"
server_key: "b292dd27400e02c0bcab9320f1c70ec5"
server_key_iv: "62b4e3217178181bc3cc2b3d72c2f695"
server_key_version: "encryptionkey@0"
I20260812 07:02:32.761837 15608 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:02:32.765017 15639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:32.766185 15608 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:02:32.766326 15608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "186f7554d2ce4a9a838664c036252830"
format_stamp: "Formatted at 2026-08-12 07:02:32 on dist-test-slave-lgr5"
server_key: "b292dd27400e02c0bcab9320f1c70ec5"
server_key_iv: "62b4e3217178181bc3cc2b3d72c2f695"
server_key_version: "encryptionkey@0"
I20260812 07:02:32.766494 15608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:32.782131 15608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:32.786551 15608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:32.786801 15608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:32.789323 15608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:32.789443 15608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:32.789564 15608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:32.789657 15608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:02:32 dist-test-slave-lgr5 krb5kdc[14931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518152, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:32.811225 15191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:32.789760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:34551 (local address 127.11.192.254:46435)
0812 07:02:32.789928 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:32.789934 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:32.791299 (+ 1365us) server_negotiation.cc:407] Connection header received
0812 07:02:32.793419 (+ 2120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:32.793427 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:32.793532 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:32.793655 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:32.795633 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.796842 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.799153 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.799710 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.799824 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:32.803955 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:32.803962 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:32.803966 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:32.804046 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:32.807486 (+ 3440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:32.808439 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:32.808445 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:32.808448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:32.808641 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:32.809180 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:32.809185 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:32.809189 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:32.809575 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:32.809724 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:32.810630 (+ 906us) server_negotiation.cc:299] Negotiation successful
0812 07:02:32.810859 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 07:02:32.812518 15664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:32.790389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:46435 (local address 127.11.192.195:34551)
0812 07:02:32.791151 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:32.791185 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:32.793065 (+ 1880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:32.793915 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:32.793925 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:32.794641 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:32.795411 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:32.795443 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.797039 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.797045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:32.799013 (+ 1968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:32.799025 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.799949 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.799957 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:32.800222 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:32.800889 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:32.800912 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:32.803657 (+ 2745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:32.807624 (+ 3967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:32.807636 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:32.807652 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:32.808269 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:32.808779 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:32.808784 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:32.808788 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:32.809037 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:32.809738 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:32.809757 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:32.810122 (+ 365us) client_negotiation.cc:769] Sending connection context
0812 07:02:32.810514 (+ 392us) client_negotiation.cc:240] Negotiation successful
0812 07:02:32.811034 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":330,"threads_started":1}
I20260812 07:02:32.827546 15608 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:36471
I20260812 07:02:32.827608 15873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:36471 every 8 connection(s)
I20260812 07:02:32.828860 15608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:02:32.829409 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 15608
I20260812 07:02:32.829609 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:02:32.830202 12035 external_mini_cluster.cc:1428] Setting key 98b8f70d6a2428ea9681b90adbed24ef
I20260812 07:02:32.830957 15875 heartbeater.cc:347] Connected to a master server at 127.11.192.254:46435
I20260812 07:02:32.831228 15875 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:32.832214 15875 heartbeater.cc:511] Master 127.11.192.254:46435 requested a full tablet report, sending...
I20260812 07:02:32.833137 15831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:32.818894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:46435 (local address 127.11.192.195:60251)
0812 07:02:32.819545 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:32.819563 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:32.819689 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:32.820163 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:32.820167 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:32.820761 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:32.821174 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:32.821192 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.822499 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.822503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:32.824035 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:32.824047 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.824808 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.824817 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:32.825005 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:32.825564 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:32.825586 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:32.826620 (+ 1034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:32.830100 (+ 3480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:32.830104 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:32.830108 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:32.830600 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:32.831083 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:32.831087 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:32.831091 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:32.831244 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:32.831971 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:32.831977 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:32.832205 (+ 228us) client_negotiation.cc:769] Sending connection context
0812 07:02:32.832344 (+ 139us) client_negotiation.cc:240] Negotiation successful
0812 07:02:32.832642 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":352,"threads_started":1}
I20260812 07:02:32.833981 15191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:32.818912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:60251 (local address 127.11.192.254:46435)
0812 07:02:32.819098 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:32.819104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:32.819642 (+ 538us) server_negotiation.cc:407] Connection header received
0812 07:02:32.819807 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:32.819813 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:32.819876 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:32.819961 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:32.821364 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.822326 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.824180 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.824601 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.824731 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:32.826805 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:32.826811 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:32.826816 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:32.826852 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:32.829933 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:32.830733 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:32.830739 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:32.830743 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:32.830937 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:32.831386 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:32.831392 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:32.831394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:32.831738 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:32.831885 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:32.833426 (+ 1541us) server_negotiation.cc:299] Negotiation successful
0812 07:02:32.833655 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:02:32.834160 15010 ts_manager.cc:194] Registered new tserver with Master: 186f7554d2ce4a9a838664c036252830 (127.11.192.195:36471)
I20260812 07:02:32.835362 15010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:34551
I20260812 07:02:32.845417 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 07:02:32 dist-test-slave-lgr5 krb5kdc[14931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260812 07:02:32.884030 15377 heartbeater.cc:503] Master 127.11.192.254:46435 was elected leader, sending a full tablet report...
Aug 12 07:02:32 dist-test-slave-lgr5 krb5kdc[14931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:32.893966 15191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:32.880390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45882 (local address 127.11.192.254:46435)
0812 07:02:32.880594 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:32.880601 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:32.880880 (+ 279us) server_negotiation.cc:407] Connection header received
0812 07:02:32.881034 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:32.881038 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:32.881100 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:32.881179 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:32.882434 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.883158 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.884452 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.885012 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.885134 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:32.888777 (+ 3643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:32.888782 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:32.888785 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:32.888817 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:32.891230 (+ 2413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:32.891844 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:32.891848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:32.891851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:32.891981 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:32.892516 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:32.892520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:32.892523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:32.892821 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:32.893050 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:32.893375 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 07:02:32.893609 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:02:32.897929 15010 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45882:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:02:32.904708 15010 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:02:32.924693 15908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:32.915923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40660 (local address 127.11.192.193:34835)
0812 07:02:32.916672 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:32.916680 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:32.916708 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:02:32.916812 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:32.916819 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:32.917012 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:32.917188 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:32.917861 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.919360 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.922052 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.923534 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.923712 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:32.923886 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:32.924233 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 07:02:32.924387 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":465,"threads_started":1}
I20260812 07:02:32.924724 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:32.915682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:34835 (local address 127.0.0.1:40660)
0812 07:02:32.916364 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:32.916402 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:32.916563 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:32.917215 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:32.917222 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:32.917251 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:32.917669 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:32.917683 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.919520 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.919526 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:32.921865 (+ 2339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:32.921880 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.923756 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.923762 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:32.923966 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:32.924031 (+ 65us) client_negotiation.cc:769] Sending connection context
0812 07:02:32.924190 (+ 159us) client_negotiation.cc:240] Negotiation successful
0812 07:02:32.924373 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":267,"threads_started":1}
I20260812 07:02:32.926600 15912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:32.917158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41620 (local address 127.11.192.195:36471)
0812 07:02:32.917859 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:32.917865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:32.918830 (+ 965us) server_negotiation.cc:407] Connection header received
0812 07:02:32.918940 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:32.918948 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:32.919142 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:32.919312 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:32.920628 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.921741 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.923757 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.925188 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.925349 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:32.925525 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:32.926092 (+ 567us) server_negotiation.cc:299] Negotiation successful
0812 07:02:32.926254 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":350,"threads_started":1}
I20260812 07:02:32.927198 15913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:32.918345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:34241 (local address 127.0.0.1:39242)
0812 07:02:32.918889 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:32.918910 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:32.919013 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:32.920007 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:32.920011 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:32.920039 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:32.920689 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:32.920702 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.922412 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.922421 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:32.924517 (+ 2096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:32.924533 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.926315 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.926321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:32.926487 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:32.926518 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 07:02:32.926683 (+ 165us) client_negotiation.cc:240] Negotiation successful
0812 07:02:32.926803 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":1,"thread_start_us":257,"threads_started":1}
I20260812 07:02:32.927459 15910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:32.916725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:36471 (local address 127.0.0.1:41620)
0812 07:02:32.918164 (+ 1439us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:32.918177 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:32.918295 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:32.919839 (+ 1544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:32.919844 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:32.919873 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:32.920361 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:32.920378 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.921912 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.921917 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:32.923574 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:32.923587 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.925707 (+ 2120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.925713 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:32.925884 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:32.925914 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 07:02:32.926901 (+ 987us) client_negotiation.cc:240] Negotiation successful
0812 07:02:32.927023 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1351,"mutex_wait_us":3,"thread_start_us":1260,"threads_started":1}
I20260812 07:02:32.927961 15917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:32.918606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39242 (local address 127.11.192.194:34241)
0812 07:02:32.919586 (+ 980us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:32.919592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:32.919612 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:02:32.919681 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:32.919686 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:32.919825 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:32.919977 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:32.920879 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.922232 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.924705 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:32.926099 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:32.926354 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:32.926512 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:32.927488 (+ 976us) server_negotiation.cc:299] Negotiation successful
0812 07:02:32.927659 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":816,"thread_start_us":991,"threads_started":1}
I20260812 07:02:32.928953 15257 tablet_service.cc:1467] Processing CreateTablet for tablet 6a0524085bcd48df919b9a0ce3dac0a0 (DEFAULT_TABLE table=test-table [id=646428a55f5c48298b86c4cc92582622]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:32.930483 15257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a0524085bcd48df919b9a0ce3dac0a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:32.930969 15742 tablet_service.cc:1467] Processing CreateTablet for tablet 6a0524085bcd48df919b9a0ce3dac0a0 (DEFAULT_TABLE table=test-table [id=646428a55f5c48298b86c4cc92582622]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:32.931020 15487 tablet_service.cc:1467] Processing CreateTablet for tablet 6a0524085bcd48df919b9a0ce3dac0a0 (DEFAULT_TABLE table=test-table [id=646428a55f5c48298b86c4cc92582622]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:32.932350 15487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a0524085bcd48df919b9a0ce3dac0a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:32.932343 15742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a0524085bcd48df919b9a0ce3dac0a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:32.942071 15920 tablet_bootstrap.cc:492] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5: Bootstrap starting.
I20260812 07:02:32.945143 15920 tablet_bootstrap.cc:654] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:32.946156 15923 tablet_bootstrap.cc:492] T 6a0524085bcd48df919b9a0ce3dac0a0 P 3cd38f56505e46dc8b8109aad1486cd4: Bootstrap starting.
I20260812 07:02:32.946336 15920 log.cc:824] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:32.946964 15922 tablet_bootstrap.cc:492] T 6a0524085bcd48df919b9a0ce3dac0a0 P 186f7554d2ce4a9a838664c036252830: Bootstrap starting.
I20260812 07:02:32.950484 15922 tablet_bootstrap.cc:654] T 6a0524085bcd48df919b9a0ce3dac0a0 P 186f7554d2ce4a9a838664c036252830: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:32.951736 15922 log.cc:824] T 6a0524085bcd48df919b9a0ce3dac0a0 P 186f7554d2ce4a9a838664c036252830: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:32.954458 15922 tablet_bootstrap.cc:492] T 6a0524085bcd48df919b9a0ce3dac0a0 P 186f7554d2ce4a9a838664c036252830: No bootstrap required, opened a new log
I20260812 07:02:32.954699 15922 ts_tablet_manager.cc:1397] T 6a0524085bcd48df919b9a0ce3dac0a0 P 186f7554d2ce4a9a838664c036252830: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 07:02:32.954922 15920 tablet_bootstrap.cc:492] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5: No bootstrap required, opened a new log
I20260812 07:02:32.955217 15920 ts_tablet_manager.cc:1397] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260812 07:02:32.955220 15923 tablet_bootstrap.cc:654] T 6a0524085bcd48df919b9a0ce3dac0a0 P 3cd38f56505e46dc8b8109aad1486cd4: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:32.956413 15923 log.cc:824] T 6a0524085bcd48df919b9a0ce3dac0a0 P 3cd38f56505e46dc8b8109aad1486cd4: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:32.959276 15923 tablet_bootstrap.cc:492] T 6a0524085bcd48df919b9a0ce3dac0a0 P 3cd38f56505e46dc8b8109aad1486cd4: No bootstrap required, opened a new log
I20260812 07:02:32.959529 15923 ts_tablet_manager.cc:1397] T 6a0524085bcd48df919b9a0ce3dac0a0 P 3cd38f56505e46dc8b8109aad1486cd4: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.005s
I20260812 07:02:32.962152 15922 raft_consensus.cc:348] T 6a0524085bcd48df919b9a0ce3dac0a0 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } }
I20260812 07:02:32.962474 15922 raft_consensus.cc:374] T 6a0524085bcd48df919b9a0ce3dac0a0 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:32.962580 15922 raft_consensus.cc:729] T 6a0524085bcd48df919b9a0ce3dac0a0 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 186f7554d2ce4a9a838664c036252830, State: Initialized, Role: FOLLOWER
I20260812 07:02:32.963120 15922 consensus_queue.cc:260] T 6a0524085bcd48df919b9a0ce3dac0a0 P 186f7554d2ce4a9a838664c036252830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } }
I20260812 07:02:32.964155 15875 heartbeater.cc:503] Master 127.11.192.254:46435 was elected leader, sending a full tablet report...
I20260812 07:02:32.964793 15922 ts_tablet_manager.cc:1428] T 6a0524085bcd48df919b9a0ce3dac0a0 P 186f7554d2ce4a9a838664c036252830: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260812 07:02:32.964916 15920 raft_consensus.cc:348] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } }
I20260812 07:02:32.965296 15920 raft_consensus.cc:374] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:32.965440 15920 raft_consensus.cc:729] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80bc5212b150451b93f08a4ff60cacc5, State: Initialized, Role: FOLLOWER
I20260812 07:02:32.965974 15920 consensus_queue.cc:260] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } }
I20260812 07:02:32.968894 15920 ts_tablet_manager.cc:1428] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260812 07:02:32.969533 15923 raft_consensus.cc:348] T 6a0524085bcd48df919b9a0ce3dac0a0 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } }
I20260812 07:02:32.969949 15923 raft_consensus.cc:374] T 6a0524085bcd48df919b9a0ce3dac0a0 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:32.970054 15923 raft_consensus.cc:729] T 6a0524085bcd48df919b9a0ce3dac0a0 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cd38f56505e46dc8b8109aad1486cd4, State: Initialized, Role: FOLLOWER
I20260812 07:02:32.970582 15923 consensus_queue.cc:260] T 6a0524085bcd48df919b9a0ce3dac0a0 P 3cd38f56505e46dc8b8109aad1486cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } }
I20260812 07:02:32.971567 15595 heartbeater.cc:503] Master 127.11.192.254:46435 was elected leader, sending a full tablet report...
I20260812 07:02:32.972235 15923 ts_tablet_manager.cc:1428] T 6a0524085bcd48df919b9a0ce3dac0a0 P 3cd38f56505e46dc8b8109aad1486cd4: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
W20260812 07:02:33.084376 15876 tablet.cc:2369] T 6a0524085bcd48df919b9a0ce3dac0a0 P 186f7554d2ce4a9a838664c036252830: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:02:33.108924 15599 tablet.cc:2369] T 6a0524085bcd48df919b9a0ce3dac0a0 P 3cd38f56505e46dc8b8109aad1486cd4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:02:33.127449 15378 tablet.cc:2369] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:02:33.146674 15931 raft_consensus.cc:482] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:02:33.146955 15931 raft_consensus.cc:504] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } }
I20260812 07:02:33.148886 15931 leader_election.cc:288] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 186f7554d2ce4a9a838664c036252830 (127.11.192.195:36471), 3cd38f56505e46dc8b8109aad1486cd4 (127.11.192.194:34241)
I20260812 07:02:33.156309 15912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:33.149256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:37975 (local address 127.11.192.195:36471)
0812 07:02:33.149438 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:33.149442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:33.150250 (+ 808us) server_negotiation.cc:407] Connection header received
0812 07:02:33.150605 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:33.150613 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:33.150708 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:33.150816 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:33.151624 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.152509 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.154515 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.155298 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.155420 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:33.155473 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:33.155883 (+ 410us) server_negotiation.cc:299] Negotiation successful
0812 07:02:33.156020 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:02:33.156354 15917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:33.150580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:46423 (local address 127.11.192.194:34241)
0812 07:02:33.150788 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:33.150794 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:33.151002 (+ 208us) server_negotiation.cc:407] Connection header received
0812 07:02:33.151160 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:33.151164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:33.151222 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:33.151294 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:33.151955 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.152891 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.154592 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.155442 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.155611 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:33.155694 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:33.155932 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 07:02:33.156061 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 07:02:33.156569 15938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:33.150308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:34241 (local address 127.11.192.193:46423)
0812 07:02:33.150903 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:33.150916 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:33.151038 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:33.151330 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:33.151335 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:33.151358 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:33.151763 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:33.151776 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.153036 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.153040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:33.154415 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:33.154429 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.155659 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.155664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:33.155776 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:33.155798 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 07:02:33.155886 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 07:02:33.156048 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":23,"thread_start_us":306,"threads_started":1}
I20260812 07:02:33.156735 15935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:33.149368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:36471 (local address 127.11.192.193:37975)
0812 07:02:33.150141 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:33.150161 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:33.150386 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:33.151077 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:33.151083 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:33.151113 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:33.151462 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:33.151474 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.152641 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.152647 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:33.154352 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:33.154366 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.155524 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.155532 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:33.155714 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:33.155749 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 07:02:33.155836 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 07:02:33.156014 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"mutex_wait_us":23,"thread_start_us":409,"threads_started":1}
I20260812 07:02:33.157526 15529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a0524085bcd48df919b9a0ce3dac0a0" candidate_uuid: "80bc5212b150451b93f08a4ff60cacc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cd38f56505e46dc8b8109aad1486cd4" is_pre_election: true
I20260812 07:02:33.157752 15779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a0524085bcd48df919b9a0ce3dac0a0" candidate_uuid: "80bc5212b150451b93f08a4ff60cacc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "186f7554d2ce4a9a838664c036252830" is_pre_election: true
I20260812 07:02:33.158142 15529 raft_consensus.cc:2454] T 6a0524085bcd48df919b9a0ce3dac0a0 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80bc5212b150451b93f08a4ff60cacc5 in term 0.
I20260812 07:02:33.158288 15779 raft_consensus.cc:2454] T 6a0524085bcd48df919b9a0ce3dac0a0 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80bc5212b150451b93f08a4ff60cacc5 in term 0.
I20260812 07:02:33.158794 15181 leader_election.cc:302] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cd38f56505e46dc8b8109aad1486cd4, 80bc5212b150451b93f08a4ff60cacc5; no voters:
I20260812 07:02:33.159134 15931 raft_consensus.cc:2781] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:02:33.159267 15931 raft_consensus.cc:482] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:02:33.159355 15931 raft_consensus.cc:3037] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:33.161578 15931 raft_consensus.cc:504] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } }
I20260812 07:02:33.162143 15931 leader_election.cc:288] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5 [CANDIDATE]: Term 1 election: Requested vote from peers 186f7554d2ce4a9a838664c036252830 (127.11.192.195:36471), 3cd38f56505e46dc8b8109aad1486cd4 (127.11.192.194:34241)
I20260812 07:02:33.162531 15779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a0524085bcd48df919b9a0ce3dac0a0" candidate_uuid: "80bc5212b150451b93f08a4ff60cacc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "186f7554d2ce4a9a838664c036252830"
I20260812 07:02:33.162599 15529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a0524085bcd48df919b9a0ce3dac0a0" candidate_uuid: "80bc5212b150451b93f08a4ff60cacc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cd38f56505e46dc8b8109aad1486cd4"
I20260812 07:02:33.162690 15779 raft_consensus.cc:3037] T 6a0524085bcd48df919b9a0ce3dac0a0 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:33.162794 15529 raft_consensus.cc:3037] T 6a0524085bcd48df919b9a0ce3dac0a0 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:33.164891 15779 raft_consensus.cc:2454] T 6a0524085bcd48df919b9a0ce3dac0a0 P 186f7554d2ce4a9a838664c036252830 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80bc5212b150451b93f08a4ff60cacc5 in term 1.
I20260812 07:02:33.164934 15529 raft_consensus.cc:2454] T 6a0524085bcd48df919b9a0ce3dac0a0 P 3cd38f56505e46dc8b8109aad1486cd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80bc5212b150451b93f08a4ff60cacc5 in term 1.
I20260812 07:02:33.165241 15181 leader_election.cc:302] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 186f7554d2ce4a9a838664c036252830, 80bc5212b150451b93f08a4ff60cacc5; no voters:
I20260812 07:02:33.165491 15931 raft_consensus.cc:2781] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:02:33.165865 15931 raft_consensus.cc:686] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5 [term 1 LEADER]: Becoming Leader. State: Replica: 80bc5212b150451b93f08a4ff60cacc5, State: Running, Role: LEADER
I20260812 07:02:33.166301 15931 consensus_queue.cc:237] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } }
I20260812 07:02:33.170812 15009 catalog_manager.cc:5495] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 80bc5212b150451b93f08a4ff60cacc5 (127.11.192.193). New cstate: current_term: 1 leader_uuid: "80bc5212b150451b93f08a4ff60cacc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } health_report { overall_health: UNKNOWN } } }
Aug 12 07:02:33 dist-test-slave-lgr5 krb5kdc[14931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518151, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:33.246763 15191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:33.234168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45894 (local address 127.11.192.254:46435)
0812 07:02:33.234321 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:33.234326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:33.234865 (+ 539us) server_negotiation.cc:407] Connection header received
0812 07:02:33.235022 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:33.235027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:33.235122 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:33.235204 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:33.236368 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.237074 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.238332 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.238772 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.238882 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:33.241994 (+ 3112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:33.241999 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:33.242003 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:33.242031 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:33.244238 (+ 2207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:33.244864 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:33.244870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:33.244873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:33.245002 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:33.245329 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:33.245334 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:33.245337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:33.245611 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:33.245741 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:33.246168 (+ 427us) server_negotiation.cc:299] Negotiation successful
0812 07:02:33.246442 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":54}
I20260812 07:02:33.246783 15956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:33.234223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:46435 (local address 127.0.0.1:45894)
0812 07:02:33.234768 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:33.234783 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:33.234914 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:33.235249 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:33.235253 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:33.235928 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:33.236228 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:33.236240 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.237196 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.237202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:33.238204 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:33.238217 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.238939 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.238946 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:33.239092 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:33.239659 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:33.239676 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:33.241816 (+ 2140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:33.244356 (+ 2540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:33.244368 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:33.244376 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:33.244759 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:33.245100 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:33.245105 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:33.245109 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:33.245234 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:33.245731 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:33.245739 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:33.245905 (+ 166us) client_negotiation.cc:769] Sending connection context
0812 07:02:33.246056 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 07:02:33.246315 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":292,"threads_started":1}
I20260812 07:02:33.251353 15009 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:45894:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 07:02:33.252351 15009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:02:33.258448 15257 tablet_service.cc:1467] Processing CreateTablet for tablet 98e08884038a40e28b914a0c47259ad2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a2bf6258678048e1be66d2fd32fec947]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:02:33.258625 15487 tablet_service.cc:1467] Processing CreateTablet for tablet 98e08884038a40e28b914a0c47259ad2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a2bf6258678048e1be66d2fd32fec947]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:02:33.258773 15742 tablet_service.cc:1467] Processing CreateTablet for tablet 98e08884038a40e28b914a0c47259ad2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a2bf6258678048e1be66d2fd32fec947]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:02:33.259439 15487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98e08884038a40e28b914a0c47259ad2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:33.259439 15257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98e08884038a40e28b914a0c47259ad2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:33.259532 15742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98e08884038a40e28b914a0c47259ad2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:33.264686 15922 tablet_bootstrap.cc:492] T 98e08884038a40e28b914a0c47259ad2 P 186f7554d2ce4a9a838664c036252830: Bootstrap starting.
I20260812 07:02:33.266276 15922 tablet_bootstrap.cc:654] T 98e08884038a40e28b914a0c47259ad2 P 186f7554d2ce4a9a838664c036252830: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:33.266407 15920 tablet_bootstrap.cc:492] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5: Bootstrap starting.
I20260812 07:02:33.266517 15923 tablet_bootstrap.cc:492] T 98e08884038a40e28b914a0c47259ad2 P 3cd38f56505e46dc8b8109aad1486cd4: Bootstrap starting.
I20260812 07:02:33.267959 15920 tablet_bootstrap.cc:654] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:33.268023 15923 tablet_bootstrap.cc:654] T 98e08884038a40e28b914a0c47259ad2 P 3cd38f56505e46dc8b8109aad1486cd4: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:33.268038 15922 tablet_bootstrap.cc:492] T 98e08884038a40e28b914a0c47259ad2 P 186f7554d2ce4a9a838664c036252830: No bootstrap required, opened a new log
I20260812 07:02:33.268232 15922 ts_tablet_manager.cc:1397] T 98e08884038a40e28b914a0c47259ad2 P 186f7554d2ce4a9a838664c036252830: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20260812 07:02:33.268842 15922 raft_consensus.cc:348] T 98e08884038a40e28b914a0c47259ad2 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:33.269200 15922 raft_consensus.cc:374] T 98e08884038a40e28b914a0c47259ad2 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:33.269347 15922 raft_consensus.cc:729] T 98e08884038a40e28b914a0c47259ad2 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 186f7554d2ce4a9a838664c036252830, State: Initialized, Role: FOLLOWER
I20260812 07:02:33.269635 15922 consensus_queue.cc:260] T 98e08884038a40e28b914a0c47259ad2 P 186f7554d2ce4a9a838664c036252830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:33.270388 15920 tablet_bootstrap.cc:492] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5: No bootstrap required, opened a new log
I20260812 07:02:33.270397 15922 ts_tablet_manager.cc:1428] T 98e08884038a40e28b914a0c47259ad2 P 186f7554d2ce4a9a838664c036252830: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 07:02:33.270278 15930 tablet_replica.cc:439] T 98e08884038a40e28b914a0c47259ad2 P 186f7554d2ce4a9a838664c036252830: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } } }
I20260812 07:02:33.270704 15930 tablet_replica.cc:442] T 98e08884038a40e28b914a0c47259ad2 P 186f7554d2ce4a9a838664c036252830: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:02:33.270707 15920 ts_tablet_manager.cc:1397] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 07:02:33.271571 15920 raft_consensus.cc:348] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:33.271808 15920 raft_consensus.cc:374] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:33.271910 15920 raft_consensus.cc:729] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80bc5212b150451b93f08a4ff60cacc5, State: Initialized, Role: FOLLOWER
I20260812 07:02:33.272154 15920 consensus_queue.cc:260] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:33.272579 15931 tablet_replica.cc:439] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } } }
I20260812 07:02:33.272764 15920 ts_tablet_manager.cc:1428] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 07:02:33.272871 15931 tablet_replica.cc:442] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:02:33.273137 15923 tablet_bootstrap.cc:492] T 98e08884038a40e28b914a0c47259ad2 P 3cd38f56505e46dc8b8109aad1486cd4: No bootstrap required, opened a new log
I20260812 07:02:33.273268 15923 ts_tablet_manager.cc:1397] T 98e08884038a40e28b914a0c47259ad2 P 3cd38f56505e46dc8b8109aad1486cd4: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 07:02:33.273945 15923 raft_consensus.cc:348] T 98e08884038a40e28b914a0c47259ad2 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:33.274143 15923 raft_consensus.cc:374] T 98e08884038a40e28b914a0c47259ad2 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:33.274238 15923 raft_consensus.cc:729] T 98e08884038a40e28b914a0c47259ad2 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cd38f56505e46dc8b8109aad1486cd4, State: Initialized, Role: FOLLOWER
I20260812 07:02:33.274462 15923 consensus_queue.cc:260] T 98e08884038a40e28b914a0c47259ad2 P 3cd38f56505e46dc8b8109aad1486cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:33.275065 15923 ts_tablet_manager.cc:1428] T 98e08884038a40e28b914a0c47259ad2 P 3cd38f56505e46dc8b8109aad1486cd4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 07:02:33.274902 15933 tablet_replica.cc:439] T 98e08884038a40e28b914a0c47259ad2 P 3cd38f56505e46dc8b8109aad1486cd4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } } }
I20260812 07:02:33.275297 15933 tablet_replica.cc:442] T 98e08884038a40e28b914a0c47259ad2 P 3cd38f56505e46dc8b8109aad1486cd4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:02:33.515017 15931 raft_consensus.cc:482] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:02:33.515250 15931 raft_consensus.cc:504] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:33.515831 15931 leader_election.cc:288] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3cd38f56505e46dc8b8109aad1486cd4 (127.11.192.194:34241), 186f7554d2ce4a9a838664c036252830 (127.11.192.195:36471)
I20260812 07:02:33.516419 15529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98e08884038a40e28b914a0c47259ad2" candidate_uuid: "80bc5212b150451b93f08a4ff60cacc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cd38f56505e46dc8b8109aad1486cd4" is_pre_election: true
I20260812 07:02:33.516494 15779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98e08884038a40e28b914a0c47259ad2" candidate_uuid: "80bc5212b150451b93f08a4ff60cacc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "186f7554d2ce4a9a838664c036252830" is_pre_election: true
I20260812 07:02:33.516666 15529 raft_consensus.cc:2454] T 98e08884038a40e28b914a0c47259ad2 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80bc5212b150451b93f08a4ff60cacc5 in term 0.
I20260812 07:02:33.516736 15779 raft_consensus.cc:2454] T 98e08884038a40e28b914a0c47259ad2 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80bc5212b150451b93f08a4ff60cacc5 in term 0.
I20260812 07:02:33.517089 15181 leader_election.cc:302] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cd38f56505e46dc8b8109aad1486cd4, 80bc5212b150451b93f08a4ff60cacc5; no voters:
I20260812 07:02:33.517356 15931 raft_consensus.cc:2781] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:02:33.517464 15931 raft_consensus.cc:482] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:02:33.517578 15931 raft_consensus.cc:3037] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:33.519470 15931 raft_consensus.cc:504] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:33.519977 15931 leader_election.cc:288] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5 [CANDIDATE]: Term 1 election: Requested vote from peers 3cd38f56505e46dc8b8109aad1486cd4 (127.11.192.194:34241), 186f7554d2ce4a9a838664c036252830 (127.11.192.195:36471)
I20260812 07:02:33.520416 15529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98e08884038a40e28b914a0c47259ad2" candidate_uuid: "80bc5212b150451b93f08a4ff60cacc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cd38f56505e46dc8b8109aad1486cd4"
I20260812 07:02:33.520486 15779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98e08884038a40e28b914a0c47259ad2" candidate_uuid: "80bc5212b150451b93f08a4ff60cacc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "186f7554d2ce4a9a838664c036252830"
I20260812 07:02:33.520612 15529 raft_consensus.cc:3037] T 98e08884038a40e28b914a0c47259ad2 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:33.520684 15779 raft_consensus.cc:3037] T 98e08884038a40e28b914a0c47259ad2 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:33.522475 15529 raft_consensus.cc:2454] T 98e08884038a40e28b914a0c47259ad2 P 3cd38f56505e46dc8b8109aad1486cd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80bc5212b150451b93f08a4ff60cacc5 in term 1.
I20260812 07:02:33.522563 15779 raft_consensus.cc:2454] T 98e08884038a40e28b914a0c47259ad2 P 186f7554d2ce4a9a838664c036252830 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80bc5212b150451b93f08a4ff60cacc5 in term 1.
I20260812 07:02:33.522881 15181 leader_election.cc:302] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cd38f56505e46dc8b8109aad1486cd4, 80bc5212b150451b93f08a4ff60cacc5; no voters:
I20260812 07:02:33.523118 15931 raft_consensus.cc:2781] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:02:33.523281 15931 raft_consensus.cc:686] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5 [term 1 LEADER]: Becoming Leader. State: Replica: 80bc5212b150451b93f08a4ff60cacc5, State: Running, Role: LEADER
I20260812 07:02:33.523512 15931 consensus_queue.cc:237] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:33.524924 15942 tablet_replica.cc:439] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5: TxnStatusTablet state changed. Reason: New leader 80bc5212b150451b93f08a4ff60cacc5. Latest consensus state: current_term: 1 leader_uuid: "80bc5212b150451b93f08a4ff60cacc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } } }
I20260812 07:02:33.525146 15942 tablet_replica.cc:442] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:02:33.525774 15966 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 07:02:33.526044 15966 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 07:02:33.526382 15006 catalog_manager.cc:5495] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 80bc5212b150451b93f08a4ff60cacc5 (127.11.192.193). New cstate: current_term: 1 leader_uuid: "80bc5212b150451b93f08a4ff60cacc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } health_report { overall_health: UNKNOWN } } }
I20260812 07:02:33.572644 15956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:33.566568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:34835 (local address 127.0.0.1:40672)
0812 07:02:33.566792 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:33.566813 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:33.566965 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:33.567569 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:33.567573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:33.567586 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:02:33.567788 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:33.567801 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.568949 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.568955 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:33.570034 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:33.570062 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.570889 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.570895 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:33.571080 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:33.571135 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:02:33.572055 (+ 920us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:02:33.572107 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 07:02:33.572186 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 07:02:33.572337 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 07:02:33.572731 15971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:33.566627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40672 (local address 127.11.192.193:34835)
0812 07:02:33.567208 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:33.567212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:33.567231 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:02:33.567313 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:33.567316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:33.567388 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:33.567535 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:33.567954 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.568822 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.570181 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.570723 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.570842 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:33.571348 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:33.571943 (+ 595us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:33.572099 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:33.572221 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 07:02:33.572301 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":254,"threads_started":1}
I20260812 07:02:33.581152 15779 raft_consensus.cc:1261] T 98e08884038a40e28b914a0c47259ad2 P 186f7554d2ce4a9a838664c036252830 [term 1 FOLLOWER]: Refusing update from remote peer 80bc5212b150451b93f08a4ff60cacc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:33.581254 15529 raft_consensus.cc:1261] T 98e08884038a40e28b914a0c47259ad2 P 3cd38f56505e46dc8b8109aad1486cd4 [term 1 FOLLOWER]: Refusing update from remote peer 80bc5212b150451b93f08a4ff60cacc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:33.582078 15931 consensus_queue.cc:1035] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:02:33.582392 15942 consensus_queue.cc:1035] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:02:33.599320 15933 tablet_replica.cc:439] T 98e08884038a40e28b914a0c47259ad2 P 3cd38f56505e46dc8b8109aad1486cd4: TxnStatusTablet state changed. Reason: New leader 80bc5212b150451b93f08a4ff60cacc5. Latest consensus state: current_term: 1 leader_uuid: "80bc5212b150451b93f08a4ff60cacc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } } }
I20260812 07:02:33.599619 15933 tablet_replica.cc:442] T 98e08884038a40e28b914a0c47259ad2 P 3cd38f56505e46dc8b8109aad1486cd4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:02:33.602173 15930 tablet_replica.cc:439] T 98e08884038a40e28b914a0c47259ad2 P 186f7554d2ce4a9a838664c036252830: TxnStatusTablet state changed. Reason: New leader 80bc5212b150451b93f08a4ff60cacc5. Latest consensus state: current_term: 1 leader_uuid: "80bc5212b150451b93f08a4ff60cacc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } } }
I20260812 07:02:33.602427 15930 tablet_replica.cc:442] T 98e08884038a40e28b914a0c47259ad2 P 186f7554d2ce4a9a838664c036252830: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:02:33.602702 15972 mvcc.cc:205] Tried to move back new op lower bound from 7317578357058752512 to 7317578356834426880. Current Snapshot: MvccSnapshot[applied={T|T < 7317578357058752512}]
I20260812 07:02:33.605974 15942 tablet_replica.cc:439] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "80bc5212b150451b93f08a4ff60cacc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } } }
I20260812 07:02:33.606227 15942 tablet_replica.cc:442] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:02:33.606482 15942 tablet_replica.cc:439] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "80bc5212b150451b93f08a4ff60cacc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } } }
I20260812 07:02:33.606747 15942 tablet_replica.cc:442] T 98e08884038a40e28b914a0c47259ad2 P 80bc5212b150451b93f08a4ff60cacc5: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:02:33.609220 15933 tablet_replica.cc:439] T 98e08884038a40e28b914a0c47259ad2 P 3cd38f56505e46dc8b8109aad1486cd4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "80bc5212b150451b93f08a4ff60cacc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } } }
I20260812 07:02:33.609447 15933 tablet_replica.cc:442] T 98e08884038a40e28b914a0c47259ad2 P 3cd38f56505e46dc8b8109aad1486cd4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:02:33.610882 15976 mvcc.cc:205] Tried to move back new op lower bound from 7317578357058752512 to 7317578356834426880. Current Snapshot: MvccSnapshot[applied={T|T < 7317578357058752512 or (T in {7317578357058752512})}]
I20260812 07:02:33.615258 15930 tablet_replica.cc:439] T 98e08884038a40e28b914a0c47259ad2 P 186f7554d2ce4a9a838664c036252830: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "80bc5212b150451b93f08a4ff60cacc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } } }
I20260812 07:02:33.615504 15930 tablet_replica.cc:442] T 98e08884038a40e28b914a0c47259ad2 P 186f7554d2ce4a9a838664c036252830: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:02:33.625682 15971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:33.615780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40676 (local address 127.11.192.193:34835)
0812 07:02:33.615975 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:33.615983 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:33.620679 (+ 4696us) server_negotiation.cc:407] Connection header received
0812 07:02:33.620843 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:33.620847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:33.620909 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:33.620998 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:33.621437 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.622146 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.624120 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.624526 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.624623 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:33.625073 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:33.625166 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:33.625226 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:33.625361 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 07:02:33.625421 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:02:33.642035 15971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:33.636429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:60281 (local address 127.11.192.193:34835)
0812 07:02:33.636568 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:33.636572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:33.637311 (+ 739us) server_negotiation.cc:407] Connection header received
0812 07:02:33.637536 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:33.637539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:33.637598 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:33.637665 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:33.638159 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.638906 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.640357 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.640809 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.640917 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:33.641356 (+ 439us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:33.641444 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:33.641527 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:33.641637 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 07:02:33.641710 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 07:02:33.642231 15991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:33.636566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:34835 (local address 127.11.192.193:60281)
0812 07:02:33.637225 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:33.637246 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:33.637382 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:33.637695 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:33.637699 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:33.637709 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:02:33.638014 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:33.638023 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.639040 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.639045 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:33.640236 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:33.640248 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.640985 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.640999 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:33.641124 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:33.641172 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:02:33.641534 (+ 362us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:02:33.641545 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 07:02:33.641611 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 07:02:33.641743 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":311,"threads_started":1}
I20260812 07:02:33.650650 15779 raft_consensus.cc:1261] T 6a0524085bcd48df919b9a0ce3dac0a0 P 186f7554d2ce4a9a838664c036252830 [term 1 FOLLOWER]: Refusing update from remote peer 80bc5212b150451b93f08a4ff60cacc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:33.650758 15529 raft_consensus.cc:1261] T 6a0524085bcd48df919b9a0ce3dac0a0 P 3cd38f56505e46dc8b8109aad1486cd4 [term 1 FOLLOWER]: Refusing update from remote peer 80bc5212b150451b93f08a4ff60cacc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:33.651182 15992 consensus_queue.cc:1035] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:02:33.651587 15942 consensus_queue.cc:1035] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:02:33.716269 15006 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:02:33.716567 15006 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:02:33.720063 15006 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a: Sending DeleteTablet for 3 replicas of tablet 6a0524085bcd48df919b9a0ce3dac0a0
I20260812 07:02:33.721050 15257 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a0524085bcd48df919b9a0ce3dac0a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:33 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:40660
I20260812 07:02:33.721179 15487 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a0524085bcd48df919b9a0ce3dac0a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:33 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:39242
I20260812 07:02:33.721793 15742 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a0524085bcd48df919b9a0ce3dac0a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:33 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:41620
I20260812 07:02:33.722333 15006 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45882:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:02:33.723158 15006 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:02:33.730279 15256 tablet_service.cc:1467] Processing CreateTablet for tablet 35fa3301233f444ab08e1d511c72a882 (DEFAULT_TABLE table=test-table [id=1e2cfef39754487790a90ac7ecbe032f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:33.730468 15486 tablet_service.cc:1467] Processing CreateTablet for tablet 35fa3301233f444ab08e1d511c72a882 (DEFAULT_TABLE table=test-table [id=1e2cfef39754487790a90ac7ecbe032f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:33.730844 15256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35fa3301233f444ab08e1d511c72a882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:33.731094 15486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35fa3301233f444ab08e1d511c72a882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:33.732275 15997 tablet_replica.cc:330] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5: stopping tablet replica
I20260812 07:02:33.732712 15997 raft_consensus.cc:2229] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:02:33.733386 15997 raft_consensus.cc:2258] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:33.736227 15923 tablet_bootstrap.cc:492] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4: Bootstrap starting.
I20260812 07:02:33.736497 15741 tablet_service.cc:1467] Processing CreateTablet for tablet 35fa3301233f444ab08e1d511c72a882 (DEFAULT_TABLE table=test-table [id=1e2cfef39754487790a90ac7ecbe032f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:33.736908 15741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35fa3301233f444ab08e1d511c72a882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:33.737835 15923 tablet_bootstrap.cc:654] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:33.740350 15923 tablet_bootstrap.cc:492] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4: No bootstrap required, opened a new log
I20260812 07:02:33.740504 15923 ts_tablet_manager.cc:1397] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 07:02:33.741235 15923 raft_consensus.cc:348] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:33.741456 15923 raft_consensus.cc:374] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:33.741518 15923 raft_consensus.cc:729] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cd38f56505e46dc8b8109aad1486cd4, State: Initialized, Role: FOLLOWER
I20260812 07:02:33.741539 15997 ts_tablet_manager.cc:1905] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:33.741959 15923 consensus_queue.cc:260] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:33.744127 15923 ts_tablet_manager.cc:1428] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260812 07:02:33.746160 15920 tablet_bootstrap.cc:492] T 35fa3301233f444ab08e1d511c72a882 P 80bc5212b150451b93f08a4ff60cacc5: Bootstrap starting.
I20260812 07:02:33.747152 15922 tablet_bootstrap.cc:492] T 35fa3301233f444ab08e1d511c72a882 P 186f7554d2ce4a9a838664c036252830: Bootstrap starting.
I20260812 07:02:33.747918 15920 tablet_bootstrap.cc:654] T 35fa3301233f444ab08e1d511c72a882 P 80bc5212b150451b93f08a4ff60cacc5: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:33.748799 15922 tablet_bootstrap.cc:654] T 35fa3301233f444ab08e1d511c72a882 P 186f7554d2ce4a9a838664c036252830: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:33.751199 15997 ts_tablet_manager.cc:1918] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:02:33.751359 15997 log.cc:1196] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-0/wal/wals/6a0524085bcd48df919b9a0ce3dac0a0
I20260812 07:02:33.751906 15997 ts_tablet_manager.cc:1939] T 6a0524085bcd48df919b9a0ce3dac0a0 P 80bc5212b150451b93f08a4ff60cacc5: Deleting consensus metadata
I20260812 07:02:33.760660 16000 tablet_replica.cc:330] T 6a0524085bcd48df919b9a0ce3dac0a0 P 186f7554d2ce4a9a838664c036252830: stopping tablet replica
I20260812 07:02:33.761683 16000 raft_consensus.cc:2229] T 6a0524085bcd48df919b9a0ce3dac0a0 P 186f7554d2ce4a9a838664c036252830 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:02:33.761860 14980 catalog_manager.cc:4841] TS 80bc5212b150451b93f08a4ff60cacc5 (127.11.192.193:34835): tablet 6a0524085bcd48df919b9a0ce3dac0a0 (table test-table [id=646428a55f5c48298b86c4cc92582622]) successfully deleted
I20260812 07:02:33.762101 16000 raft_consensus.cc:2258] T 6a0524085bcd48df919b9a0ce3dac0a0 P 186f7554d2ce4a9a838664c036252830 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:33.763381 15920 tablet_bootstrap.cc:492] T 35fa3301233f444ab08e1d511c72a882 P 80bc5212b150451b93f08a4ff60cacc5: No bootstrap required, opened a new log
I20260812 07:02:33.763619 15920 ts_tablet_manager.cc:1397] T 35fa3301233f444ab08e1d511c72a882 P 80bc5212b150451b93f08a4ff60cacc5: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.000s
I20260812 07:02:33.764066 16000 ts_tablet_manager.cc:1905] T 6a0524085bcd48df919b9a0ce3dac0a0 P 186f7554d2ce4a9a838664c036252830: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:33.764446 15920 raft_consensus.cc:348] T 35fa3301233f444ab08e1d511c72a882 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:33.764642 15999 tablet_replica.cc:330] T 6a0524085bcd48df919b9a0ce3dac0a0 P 3cd38f56505e46dc8b8109aad1486cd4: stopping tablet replica
I20260812 07:02:33.764695 15920 raft_consensus.cc:374] T 35fa3301233f444ab08e1d511c72a882 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:33.764842 15920 raft_consensus.cc:729] T 35fa3301233f444ab08e1d511c72a882 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80bc5212b150451b93f08a4ff60cacc5, State: Initialized, Role: FOLLOWER
I20260812 07:02:33.765074 15999 raft_consensus.cc:2229] T 6a0524085bcd48df919b9a0ce3dac0a0 P 3cd38f56505e46dc8b8109aad1486cd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:02:33.765178 15920 consensus_queue.cc:260] T 35fa3301233f444ab08e1d511c72a882 P 80bc5212b150451b93f08a4ff60cacc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:33.765817 15920 ts_tablet_manager.cc:1428] T 35fa3301233f444ab08e1d511c72a882 P 80bc5212b150451b93f08a4ff60cacc5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 07:02:33.766327 15999 raft_consensus.cc:2258] T 6a0524085bcd48df919b9a0ce3dac0a0 P 3cd38f56505e46dc8b8109aad1486cd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:33.766739 15922 tablet_bootstrap.cc:492] T 35fa3301233f444ab08e1d511c72a882 P 186f7554d2ce4a9a838664c036252830: No bootstrap required, opened a new log
I20260812 07:02:33.766902 15922 ts_tablet_manager.cc:1397] T 35fa3301233f444ab08e1d511c72a882 P 186f7554d2ce4a9a838664c036252830: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.004s
I20260812 07:02:33.767773 15922 raft_consensus.cc:348] T 35fa3301233f444ab08e1d511c72a882 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:33.768030 15922 raft_consensus.cc:374] T 35fa3301233f444ab08e1d511c72a882 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:33.768143 15922 raft_consensus.cc:729] T 35fa3301233f444ab08e1d511c72a882 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 186f7554d2ce4a9a838664c036252830, State: Initialized, Role: FOLLOWER
I20260812 07:02:33.768391 15922 consensus_queue.cc:260] T 35fa3301233f444ab08e1d511c72a882 P 186f7554d2ce4a9a838664c036252830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:33.768527 15999 ts_tablet_manager.cc:1905] T 6a0524085bcd48df919b9a0ce3dac0a0 P 3cd38f56505e46dc8b8109aad1486cd4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:33.768924 15922 ts_tablet_manager.cc:1428] T 35fa3301233f444ab08e1d511c72a882 P 186f7554d2ce4a9a838664c036252830: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 07:02:33.772891 16000 ts_tablet_manager.cc:1918] T 6a0524085bcd48df919b9a0ce3dac0a0 P 186f7554d2ce4a9a838664c036252830: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:02:33.773080 16000 log.cc:1196] T 6a0524085bcd48df919b9a0ce3dac0a0 P 186f7554d2ce4a9a838664c036252830: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/6a0524085bcd48df919b9a0ce3dac0a0
I20260812 07:02:33.773932 16000 ts_tablet_manager.cc:1939] T 6a0524085bcd48df919b9a0ce3dac0a0 P 186f7554d2ce4a9a838664c036252830: Deleting consensus metadata
I20260812 07:02:33.775579 14980 catalog_manager.cc:4841] TS 186f7554d2ce4a9a838664c036252830 (127.11.192.195:36471): tablet 6a0524085bcd48df919b9a0ce3dac0a0 (table test-table [id=646428a55f5c48298b86c4cc92582622]) successfully deleted
I20260812 07:02:33.778592 15999 ts_tablet_manager.cc:1918] T 6a0524085bcd48df919b9a0ce3dac0a0 P 3cd38f56505e46dc8b8109aad1486cd4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:02:33.778754 15999 log.cc:1196] T 6a0524085bcd48df919b9a0ce3dac0a0 P 3cd38f56505e46dc8b8109aad1486cd4: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/6a0524085bcd48df919b9a0ce3dac0a0
I20260812 07:02:33.779213 15999 ts_tablet_manager.cc:1939] T 6a0524085bcd48df919b9a0ce3dac0a0 P 3cd38f56505e46dc8b8109aad1486cd4: Deleting consensus metadata
I20260812 07:02:33.780499 14980 catalog_manager.cc:4841] TS 3cd38f56505e46dc8b8109aad1486cd4 (127.11.192.194:34241): tablet 6a0524085bcd48df919b9a0ce3dac0a0 (table test-table [id=646428a55f5c48298b86c4cc92582622]) successfully deleted
I20260812 07:02:33.993359 15933 raft_consensus.cc:482] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:02:33.993613 15933 raft_consensus.cc:504] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:33.995491 15933 leader_election.cc:288] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80bc5212b150451b93f08a4ff60cacc5 (127.11.192.193:34835), 186f7554d2ce4a9a838664c036252830 (127.11.192.195:36471)
I20260812 07:02:34.004552 15971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:33.995786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:37367 (local address 127.11.192.193:34835)
0812 07:02:33.996019 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:33.996026 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:33.996394 (+ 368us) server_negotiation.cc:407] Connection header received
0812 07:02:33.996729 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:33.996736 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:33.996821 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:33.996887 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:33.997732 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.998928 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:34.001631 (+ 2703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:34.003157 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:34.003296 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:34.003383 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:34.004063 (+ 680us) server_negotiation.cc:299] Negotiation successful
0812 07:02:34.004234 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:02:34.005678 16005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:33.995724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:34835 (local address 127.11.192.194:37367)
0812 07:02:33.996292 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:33.996310 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:33.996539 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:33.997095 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:33.997100 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:33.997130 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:33.997518 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:33.997530 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:33.999154 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:33.999159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:34.001420 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:34.001439 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:34.003549 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:34.003555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:34.003819 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:34.003857 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 07:02:34.005114 (+ 1257us) client_negotiation.cc:240] Negotiation successful
0812 07:02:34.005270 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":267,"threads_started":1}
I20260812 07:02:34.007368 15306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35fa3301233f444ab08e1d511c72a882" candidate_uuid: "3cd38f56505e46dc8b8109aad1486cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80bc5212b150451b93f08a4ff60cacc5" is_pre_election: true
I20260812 07:02:34.007885 15306 raft_consensus.cc:2454] T 35fa3301233f444ab08e1d511c72a882 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3cd38f56505e46dc8b8109aad1486cd4 in term 0.
I20260812 07:02:34.008584 15409 leader_election.cc:302] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cd38f56505e46dc8b8109aad1486cd4, 80bc5212b150451b93f08a4ff60cacc5; no voters:
I20260812 07:02:34.009038 15933 raft_consensus.cc:2781] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:02:34.009184 15933 raft_consensus.cc:482] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:02:34.009295 15933 raft_consensus.cc:3037] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:34.012225 15933 raft_consensus.cc:504] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:34.012907 15933 leader_election.cc:288] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4 [CANDIDATE]: Term 1 election: Requested vote from peers 80bc5212b150451b93f08a4ff60cacc5 (127.11.192.193:34835), 186f7554d2ce4a9a838664c036252830 (127.11.192.195:36471)
I20260812 07:02:34.013036 16005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:33.996432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:36471 (local address 127.11.192.194:34953)
0812 07:02:34.006086 (+ 9654us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:34.006107 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:34.006218 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:34.007287 (+ 1069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:34.007291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:34.007317 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:34.007472 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:34.007481 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:34.008705 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:34.008714 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:34.010532 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:34.010565 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:34.012259 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:34.012265 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:34.012461 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:34.012494 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:02:34.012587 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 07:02:34.012715 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9543,"thread_start_us":9992,"threads_started":1}
I20260812 07:02:34.013108 16014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:33.996640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:34953 (local address 127.11.192.195:36471)
0812 07:02:34.007012 (+ 10372us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:34.007019 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:34.007044 (+ 25us) server_negotiation.cc:407] Connection header received
0812 07:02:34.007104 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:34.007107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:34.007165 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:34.007249 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:34.007622 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:34.008572 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:34.010757 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:34.012064 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:34.012280 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:34.012356 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:34.012617 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 07:02:34.012765 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10242,"thread_start_us":10099,"threads_started":1}
I20260812 07:02:34.013633 15306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35fa3301233f444ab08e1d511c72a882" candidate_uuid: "3cd38f56505e46dc8b8109aad1486cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80bc5212b150451b93f08a4ff60cacc5"
I20260812 07:02:34.013849 15306 raft_consensus.cc:3037] T 35fa3301233f444ab08e1d511c72a882 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:34.014125 15779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35fa3301233f444ab08e1d511c72a882" candidate_uuid: "3cd38f56505e46dc8b8109aad1486cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "186f7554d2ce4a9a838664c036252830" is_pre_election: true
I20260812 07:02:34.014385 15779 raft_consensus.cc:2454] T 35fa3301233f444ab08e1d511c72a882 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3cd38f56505e46dc8b8109aad1486cd4 in term 0.
I20260812 07:02:34.014385 15778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35fa3301233f444ab08e1d511c72a882" candidate_uuid: "3cd38f56505e46dc8b8109aad1486cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "186f7554d2ce4a9a838664c036252830"
I20260812 07:02:34.014693 15778 raft_consensus.cc:3037] T 35fa3301233f444ab08e1d511c72a882 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:34.016927 15306 raft_consensus.cc:2454] T 35fa3301233f444ab08e1d511c72a882 P 80bc5212b150451b93f08a4ff60cacc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cd38f56505e46dc8b8109aad1486cd4 in term 1.
I20260812 07:02:34.017355 15409 leader_election.cc:302] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cd38f56505e46dc8b8109aad1486cd4, 80bc5212b150451b93f08a4ff60cacc5; no voters:
I20260812 07:02:34.017582 15778 raft_consensus.cc:2454] T 35fa3301233f444ab08e1d511c72a882 P 186f7554d2ce4a9a838664c036252830 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cd38f56505e46dc8b8109aad1486cd4 in term 1.
I20260812 07:02:34.017614 15933 raft_consensus.cc:2781] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:02:34.018285 15933 raft_consensus.cc:686] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4 [term 1 LEADER]: Becoming Leader. State: Replica: 3cd38f56505e46dc8b8109aad1486cd4, State: Running, Role: LEADER
I20260812 07:02:34.018735 15933 consensus_queue.cc:237] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:34.022178 15009 catalog_manager.cc:5495] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3cd38f56505e46dc8b8109aad1486cd4 (127.11.192.194). New cstate: current_term: 1 leader_uuid: "3cd38f56505e46dc8b8109aad1486cd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } health_report { overall_health: UNKNOWN } } }
I20260812 07:02:34.050297 16017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:34.043784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39258 (local address 127.11.192.194:34241)
0812 07:02:34.044503 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:34.044510 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:34.044534 (+ 24us) server_negotiation.cc:407] Connection header received
0812 07:02:34.044626 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:34.044631 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:34.044725 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:34.044892 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:34.045201 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:34.046420 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:34.047671 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:34.048367 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:34.048487 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:34.048840 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:34.049615 (+ 775us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:34.049780 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:34.049848 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 07:02:34.049929 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":469,"threads_started":1}
I20260812 07:02:34.065490 15971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:34.060156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:32809 (local address 127.11.192.193:34835)
0812 07:02:34.060298 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:34.060303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:34.060806 (+ 503us) server_negotiation.cc:407] Connection header received
0812 07:02:34.060987 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:34.060994 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:34.061059 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:34.061146 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:34.061595 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:34.062456 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:34.063733 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:34.064238 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:34.064343 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:34.064757 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:34.064860 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:34.064938 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:34.065070 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 07:02:34.065161 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 07:02:34.065534 16022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:34.060114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:34835 (local address 127.11.192.194:32809)
0812 07:02:34.060703 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:34.060721 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:34.060851 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:34.061163 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:34.061168 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:34.061179 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:02:34.061456 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:34.061464 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:34.062587 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:34.062592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:34.063598 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:34.063610 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:34.064415 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:34.064421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:34.064548 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:34.064598 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:02:34.064951 (+ 353us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:02:34.064963 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 07:02:34.065040 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 07:02:34.065166 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":303,"threads_started":1}
I20260812 07:02:34.074112 15778 raft_consensus.cc:1261] T 35fa3301233f444ab08e1d511c72a882 P 186f7554d2ce4a9a838664c036252830 [term 1 FOLLOWER]: Refusing update from remote peer 3cd38f56505e46dc8b8109aad1486cd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:34.074424 15306 raft_consensus.cc:1261] T 35fa3301233f444ab08e1d511c72a882 P 80bc5212b150451b93f08a4ff60cacc5 [term 1 FOLLOWER]: Refusing update from remote peer 3cd38f56505e46dc8b8109aad1486cd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:34.074747 16015 consensus_queue.cc:1035] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:02:34.075147 15933 consensus_queue.cc:1035] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:02:34.083669 15973 mvcc.cc:205] Tried to move back new op lower bound from 7317578359082037248 to 7317578358864367616. Current Snapshot: MvccSnapshot[applied={T|T < 7317578359082037248}]
I20260812 07:02:34.104240 16017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:34.099361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:37573 (local address 127.11.192.194:34241)
0812 07:02:34.099511 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:34.099517 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:34.099537 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:02:34.099660 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:34.099664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:34.099720 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:34.099798 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:34.100242 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:34.101119 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:34.102523 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:34.102945 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:34.103059 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:34.103442 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:34.103569 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:34.103681 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:34.103826 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 07:02:34.103905 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 07:02:34.104329 15991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:34.099181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:34241 (local address 127.11.192.193:37573)
0812 07:02:34.099422 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:34.099441 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:34.099551 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:34.099836 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:34.099841 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:34.099853 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:02:34.100035 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:34.100051 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:34.101277 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:34.101282 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:34.102386 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:34.102399 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:34.103095 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:34.103101 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:34.103237 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:34.103276 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:02:34.103699 (+ 423us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:02:34.103711 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 07:02:34.103797 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 07:02:34.103951 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
W20260812 07:02:34.105435 15976 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 07:02:35.697256 15006 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:02:35.697474 15006 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:02:35.701280 15006 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a: Sending DeleteTablet for 3 replicas of tablet 35fa3301233f444ab08e1d511c72a882
I20260812 07:02:35.701921 15256 tablet_service.cc:1514] Processing DeleteTablet for tablet 35fa3301233f444ab08e1d511c72a882 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:35 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:40660
I20260812 07:02:35.702049 15487 tablet_service.cc:1514] Processing DeleteTablet for tablet 35fa3301233f444ab08e1d511c72a882 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:35 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:39242
I20260812 07:02:35.702268 15742 tablet_service.cc:1514] Processing DeleteTablet for tablet 35fa3301233f444ab08e1d511c72a882 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:35 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:41620
I20260812 07:02:35.713124 16064 tablet_replica.cc:330] T 35fa3301233f444ab08e1d511c72a882 P 186f7554d2ce4a9a838664c036252830: stopping tablet replica
I20260812 07:02:35.713454 16064 raft_consensus.cc:2229] T 35fa3301233f444ab08e1d511c72a882 P 186f7554d2ce4a9a838664c036252830 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:02:35.713761 16064 raft_consensus.cc:2258] T 35fa3301233f444ab08e1d511c72a882 P 186f7554d2ce4a9a838664c036252830 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:35.714376 16063 tablet_replica.cc:330] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4: stopping tablet replica
I20260812 07:02:35.714571 16063 raft_consensus.cc:2229] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:02:35.714936 16064 ts_tablet_manager.cc:1905] T 35fa3301233f444ab08e1d511c72a882 P 186f7554d2ce4a9a838664c036252830: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:35.715270 16063 raft_consensus.cc:2258] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:35.716401 16063 ts_tablet_manager.cc:1905] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:35.716993 16061 tablet_replica.cc:330] T 35fa3301233f444ab08e1d511c72a882 P 80bc5212b150451b93f08a4ff60cacc5: stopping tablet replica
I20260812 07:02:35.717186 16061 raft_consensus.cc:2229] T 35fa3301233f444ab08e1d511c72a882 P 80bc5212b150451b93f08a4ff60cacc5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:02:35.717438 16061 raft_consensus.cc:2258] T 35fa3301233f444ab08e1d511c72a882 P 80bc5212b150451b93f08a4ff60cacc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:35.718324 16061 ts_tablet_manager.cc:1905] T 35fa3301233f444ab08e1d511c72a882 P 80bc5212b150451b93f08a4ff60cacc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:35.721513 16064 ts_tablet_manager.cc:1918] T 35fa3301233f444ab08e1d511c72a882 P 186f7554d2ce4a9a838664c036252830: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 07:02:35.721698 16064 log.cc:1196] T 35fa3301233f444ab08e1d511c72a882 P 186f7554d2ce4a9a838664c036252830: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/35fa3301233f444ab08e1d511c72a882
I20260812 07:02:35.721701 16063 ts_tablet_manager.cc:1918] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 07:02:35.721962 16063 log.cc:1196] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/35fa3301233f444ab08e1d511c72a882
I20260812 07:02:35.722195 16064 ts_tablet_manager.cc:1939] T 35fa3301233f444ab08e1d511c72a882 P 186f7554d2ce4a9a838664c036252830: Deleting consensus metadata
I20260812 07:02:35.722429 16063 ts_tablet_manager.cc:1939] T 35fa3301233f444ab08e1d511c72a882 P 3cd38f56505e46dc8b8109aad1486cd4: Deleting consensus metadata
I20260812 07:02:35.723376 14980 catalog_manager.cc:4841] TS 186f7554d2ce4a9a838664c036252830 (127.11.192.195:36471): tablet 35fa3301233f444ab08e1d511c72a882 (table test-table [id=1e2cfef39754487790a90ac7ecbe032f]) successfully deleted
I20260812 07:02:35.723685 14980 catalog_manager.cc:4841] TS 3cd38f56505e46dc8b8109aad1486cd4 (127.11.192.194:34241): tablet 35fa3301233f444ab08e1d511c72a882 (table test-table [id=1e2cfef39754487790a90ac7ecbe032f]) successfully deleted
I20260812 07:02:35.724421 16061 ts_tablet_manager.cc:1918] T 35fa3301233f444ab08e1d511c72a882 P 80bc5212b150451b93f08a4ff60cacc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 07:02:35.724530 16061 log.cc:1196] T 35fa3301233f444ab08e1d511c72a882 P 80bc5212b150451b93f08a4ff60cacc5: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-0/wal/wals/35fa3301233f444ab08e1d511c72a882
I20260812 07:02:35.724886 16061 ts_tablet_manager.cc:1939] T 35fa3301233f444ab08e1d511c72a882 P 80bc5212b150451b93f08a4ff60cacc5: Deleting consensus metadata
I20260812 07:02:35.725728 14980 catalog_manager.cc:4841] TS 80bc5212b150451b93f08a4ff60cacc5 (127.11.192.193:34835): tablet 35fa3301233f444ab08e1d511c72a882 (table test-table [id=1e2cfef39754487790a90ac7ecbe032f]) successfully deleted
Aug 12 07:02:35 dist-test-slave-lgr5 krb5kdc[14931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518155, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 07:02:35 dist-test-slave-lgr5 krb5kdc[14931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518155, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:35.760885 16082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:35.748152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45910 (local address 127.11.192.254:46435)
0812 07:02:35.748727 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:35.748732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:35.748755 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:02:35.748919 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:35.748923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:35.748991 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:35.749078 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:35.750292 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:35.751093 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:35.752369 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:35.752762 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:35.752877 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:35.756322 (+ 3445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:35.756327 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:35.756330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:35.756365 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:35.758722 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:35.759287 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:35.759291 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:35.759293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:35.759421 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:35.759691 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:35.759694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:35.759696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:35.759951 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:35.760190 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:35.760380 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 07:02:35.760542 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":277,"threads_started":1}
I20260812 07:02:35.764139 15006 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45910:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:02:35.764758 15006 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:02:35.770781 15256 tablet_service.cc:1467] Processing CreateTablet for tablet d5fe90d6640c4a2daad1468f93024174 (DEFAULT_TABLE table=test-table [id=8b1a565167c04ea188cdd0b8fdd7e6fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:35.771015 15742 tablet_service.cc:1467] Processing CreateTablet for tablet d5fe90d6640c4a2daad1468f93024174 (DEFAULT_TABLE table=test-table [id=8b1a565167c04ea188cdd0b8fdd7e6fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:35.771005 15487 tablet_service.cc:1467] Processing CreateTablet for tablet d5fe90d6640c4a2daad1468f93024174 (DEFAULT_TABLE table=test-table [id=8b1a565167c04ea188cdd0b8fdd7e6fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:35.771286 15256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5fe90d6640c4a2daad1468f93024174. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:35.771420 15742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5fe90d6640c4a2daad1468f93024174. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:35.771420 15487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5fe90d6640c4a2daad1468f93024174. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:35.778278 16085 tablet_bootstrap.cc:492] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830: Bootstrap starting.
I20260812 07:02:35.779183 16083 tablet_bootstrap.cc:492] T d5fe90d6640c4a2daad1468f93024174 P 80bc5212b150451b93f08a4ff60cacc5: Bootstrap starting.
I20260812 07:02:35.779513 16086 tablet_bootstrap.cc:492] T d5fe90d6640c4a2daad1468f93024174 P 3cd38f56505e46dc8b8109aad1486cd4: Bootstrap starting.
I20260812 07:02:35.780710 16083 tablet_bootstrap.cc:654] T d5fe90d6640c4a2daad1468f93024174 P 80bc5212b150451b93f08a4ff60cacc5: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:35.780824 16085 tablet_bootstrap.cc:654] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:35.781193 16086 tablet_bootstrap.cc:654] T d5fe90d6640c4a2daad1468f93024174 P 3cd38f56505e46dc8b8109aad1486cd4: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:35.782559 16083 tablet_bootstrap.cc:492] T d5fe90d6640c4a2daad1468f93024174 P 80bc5212b150451b93f08a4ff60cacc5: No bootstrap required, opened a new log
I20260812 07:02:35.782696 16083 ts_tablet_manager.cc:1397] T d5fe90d6640c4a2daad1468f93024174 P 80bc5212b150451b93f08a4ff60cacc5: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 07:02:35.783061 16086 tablet_bootstrap.cc:492] T d5fe90d6640c4a2daad1468f93024174 P 3cd38f56505e46dc8b8109aad1486cd4: No bootstrap required, opened a new log
I20260812 07:02:35.783179 16086 ts_tablet_manager.cc:1397] T d5fe90d6640c4a2daad1468f93024174 P 3cd38f56505e46dc8b8109aad1486cd4: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 07:02:35.783335 16083 raft_consensus.cc:348] T d5fe90d6640c4a2daad1468f93024174 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:35.783565 16083 raft_consensus.cc:374] T d5fe90d6640c4a2daad1468f93024174 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:35.783669 16083 raft_consensus.cc:729] T d5fe90d6640c4a2daad1468f93024174 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80bc5212b150451b93f08a4ff60cacc5, State: Initialized, Role: FOLLOWER
I20260812 07:02:35.783907 16086 raft_consensus.cc:348] T d5fe90d6640c4a2daad1468f93024174 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:35.783934 16083 consensus_queue.cc:260] T d5fe90d6640c4a2daad1468f93024174 P 80bc5212b150451b93f08a4ff60cacc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:35.784155 16086 raft_consensus.cc:374] T d5fe90d6640c4a2daad1468f93024174 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:35.784237 16086 raft_consensus.cc:729] T d5fe90d6640c4a2daad1468f93024174 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cd38f56505e46dc8b8109aad1486cd4, State: Initialized, Role: FOLLOWER
I20260812 07:02:35.784543 16083 ts_tablet_manager.cc:1428] T d5fe90d6640c4a2daad1468f93024174 P 80bc5212b150451b93f08a4ff60cacc5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 07:02:35.784476 16086 consensus_queue.cc:260] T d5fe90d6640c4a2daad1468f93024174 P 3cd38f56505e46dc8b8109aad1486cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:35.785068 16086 ts_tablet_manager.cc:1428] T d5fe90d6640c4a2daad1468f93024174 P 3cd38f56505e46dc8b8109aad1486cd4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 07:02:35.785823 16085 tablet_bootstrap.cc:492] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830: No bootstrap required, opened a new log
I20260812 07:02:35.785980 16085 ts_tablet_manager.cc:1397] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260812 07:02:35.786772 16085 raft_consensus.cc:348] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:35.787014 16085 raft_consensus.cc:374] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:35.787118 16085 raft_consensus.cc:729] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 186f7554d2ce4a9a838664c036252830, State: Initialized, Role: FOLLOWER
I20260812 07:02:35.787350 16085 consensus_queue.cc:260] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:35.788226 16085 ts_tablet_manager.cc:1428] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 07:02:35.877879 16093 raft_consensus.cc:482] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:02:35.878126 16093 raft_consensus.cc:504] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:35.879845 16093 leader_election.cc:288] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80bc5212b150451b93f08a4ff60cacc5 (127.11.192.193:34835), 3cd38f56505e46dc8b8109aad1486cd4 (127.11.192.194:34241)
I20260812 07:02:35.889669 16094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:35.880046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:34835 (local address 127.11.192.195:44407)
0812 07:02:35.880937 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:35.880955 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:35.881084 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:35.881734 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:35.881740 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:35.881769 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:35.882132 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:35.882144 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:35.883905 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:35.883911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:35.886734 (+ 2823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:35.886753 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:35.888802 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:35.888809 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:35.889009 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:35.889046 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 07:02:35.889139 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 07:02:35.889281 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"thread_start_us":595,"threads_started":1}
I20260812 07:02:35.890551 16098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:35.880152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:44407 (local address 127.11.192.193:34835)
0812 07:02:35.880754 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:35.880761 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:35.881204 (+ 443us) server_negotiation.cc:407] Connection header received
0812 07:02:35.881293 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:35.881299 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:35.881387 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:35.881496 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:35.882365 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:35.883718 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:35.886956 (+ 3238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:35.888347 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:35.888562 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:35.888645 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:35.890020 (+ 1375us) server_negotiation.cc:299] Negotiation successful
0812 07:02:35.890180 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":299,"threads_started":1}
I20260812 07:02:35.892300 15306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fe90d6640c4a2daad1468f93024174" candidate_uuid: "186f7554d2ce4a9a838664c036252830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80bc5212b150451b93f08a4ff60cacc5" is_pre_election: true
I20260812 07:02:35.892597 15306 raft_consensus.cc:2454] T d5fe90d6640c4a2daad1468f93024174 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 186f7554d2ce4a9a838664c036252830 in term 0.
I20260812 07:02:35.893172 15644 leader_election.cc:302] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 186f7554d2ce4a9a838664c036252830, 80bc5212b150451b93f08a4ff60cacc5; no voters:
I20260812 07:02:35.893599 16093 raft_consensus.cc:2781] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:02:35.893719 16093 raft_consensus.cc:482] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:02:35.893805 16093 raft_consensus.cc:3037] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:35.896067 16093 raft_consensus.cc:504] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:35.896790 16093 leader_election.cc:288] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830 [CANDIDATE]: Term 1 election: Requested vote from peers 80bc5212b150451b93f08a4ff60cacc5 (127.11.192.193:34835), 3cd38f56505e46dc8b8109aad1486cd4 (127.11.192.194:34241)
I20260812 07:02:35.897249 15306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fe90d6640c4a2daad1468f93024174" candidate_uuid: "186f7554d2ce4a9a838664c036252830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80bc5212b150451b93f08a4ff60cacc5"
I20260812 07:02:35.897462 15306 raft_consensus.cc:3037] T d5fe90d6640c4a2daad1468f93024174 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:35.898197 16094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:35.881084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:34241 (local address 127.11.192.195:35757)
0812 07:02:35.891033 (+ 9949us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:35.891051 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:35.891150 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:35.892528 (+ 1378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:35.892533 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:35.892557 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:35.892702 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:35.892712 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:35.894026 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:35.894032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:35.896007 (+ 1975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:35.896022 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:35.897447 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:35.897453 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:35.897642 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:35.897677 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 07:02:35.897764 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 07:02:35.897890 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9857,"thread_start_us":10530,"threads_started":1}
I20260812 07:02:35.898259 16102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:35.881240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:35757 (local address 127.11.192.194:34241)
0812 07:02:35.892258 (+ 11018us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:35.892262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:35.892282 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:02:35.892339 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:35.892342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:35.892396 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:35.892493 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:35.892860 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:35.893826 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:35.896200 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:35.897300 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:35.897466 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:35.897553 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:35.897823 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 07:02:35.897956 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10904,"thread_start_us":10741,"threads_started":1}
I20260812 07:02:35.899045 15528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fe90d6640c4a2daad1468f93024174" candidate_uuid: "186f7554d2ce4a9a838664c036252830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cd38f56505e46dc8b8109aad1486cd4" is_pre_election: true
I20260812 07:02:35.899314 15528 raft_consensus.cc:2454] T d5fe90d6640c4a2daad1468f93024174 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 186f7554d2ce4a9a838664c036252830 in term 0.
I20260812 07:02:35.899741 15529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fe90d6640c4a2daad1468f93024174" candidate_uuid: "186f7554d2ce4a9a838664c036252830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cd38f56505e46dc8b8109aad1486cd4"
I20260812 07:02:35.899924 15529 raft_consensus.cc:3037] T d5fe90d6640c4a2daad1468f93024174 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:35.900370 15306 raft_consensus.cc:2454] T d5fe90d6640c4a2daad1468f93024174 P 80bc5212b150451b93f08a4ff60cacc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 186f7554d2ce4a9a838664c036252830 in term 1.
I20260812 07:02:35.900717 15644 leader_election.cc:302] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 186f7554d2ce4a9a838664c036252830, 80bc5212b150451b93f08a4ff60cacc5; no voters:
I20260812 07:02:35.900997 16093 raft_consensus.cc:2781] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:02:35.902141 15529 raft_consensus.cc:2454] T d5fe90d6640c4a2daad1468f93024174 P 3cd38f56505e46dc8b8109aad1486cd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 186f7554d2ce4a9a838664c036252830 in term 1.
I20260812 07:02:35.902487 16093 raft_consensus.cc:686] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830 [term 1 LEADER]: Becoming Leader. State: Replica: 186f7554d2ce4a9a838664c036252830, State: Running, Role: LEADER
I20260812 07:02:35.902932 16093 consensus_queue.cc:237] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:35.906203 15010 catalog_manager.cc:5495] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830 reported cstate change: term changed from 0 to 1, leader changed from <none> to 186f7554d2ce4a9a838664c036252830 (127.11.192.195). New cstate: current_term: 1 leader_uuid: "186f7554d2ce4a9a838664c036252830" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } health_report { overall_health: HEALTHY } } }
I20260812 07:02:35.937458 16112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:35.932264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41634 (local address 127.11.192.195:36471)
0812 07:02:35.932941 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:35.932945 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:35.932964 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:02:35.933046 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:35.933050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:35.933129 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:35.933261 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:35.933541 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:35.934272 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:35.935480 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:35.935865 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:35.935950 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:35.936343 (+ 393us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:35.936919 (+ 576us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:35.937038 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:35.937102 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 07:02:35.937161 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":451,"threads_started":1}
I20260812 07:02:35.951115 16098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:35.946508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:43355 (local address 127.11.192.193:34835)
0812 07:02:35.946673 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:35.946677 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:35.947167 (+ 490us) server_negotiation.cc:407] Connection header received
0812 07:02:35.947304 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:35.947307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:35.947379 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:35.947456 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:35.947882 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:35.948627 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:35.949831 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:35.950202 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:35.950289 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:35.950616 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:35.950708 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:35.950770 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:35.950863 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 07:02:35.950922 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:02:35.951227 16123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:35.946410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:34835 (local address 127.11.192.195:43355)
0812 07:02:35.947083 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:35.947103 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:35.947208 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:35.947467 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:35.947471 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:35.947481 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:02:35.947760 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:35.947768 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:35.948723 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:35.948727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:35.949725 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:35.949736 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:35.950330 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:35.950335 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:35.950445 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:35.950494 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:02:35.950776 (+ 282us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:02:35.950788 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 07:02:35.950839 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 07:02:35.950929 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":346,"threads_started":1}
I20260812 07:02:35.958596 15306 raft_consensus.cc:1261] T d5fe90d6640c4a2daad1468f93024174 P 80bc5212b150451b93f08a4ff60cacc5 [term 1 FOLLOWER]: Refusing update from remote peer 186f7554d2ce4a9a838664c036252830: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:35.958632 15529 raft_consensus.cc:1261] T d5fe90d6640c4a2daad1468f93024174 P 3cd38f56505e46dc8b8109aad1486cd4 [term 1 FOLLOWER]: Refusing update from remote peer 186f7554d2ce4a9a838664c036252830: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:35.959210 16105 consensus_queue.cc:1035] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:02:35.959573 16093 consensus_queue.cc:1035] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:02:35.983558 16112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:35.978465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:44131 (local address 127.11.192.195:36471)
0812 07:02:35.978709 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:35.978715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:35.978858 (+ 143us) server_negotiation.cc:407] Connection header received
0812 07:02:35.979045 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:35.979050 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:35.979115 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:35.979189 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:35.979775 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:35.980693 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:35.982002 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:35.982416 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:35.982524 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:35.982900 (+ 376us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:35.982997 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:35.983073 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:35.983209 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 07:02:35.983280 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260812 07:02:35.983656 16131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:35.978202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:36471 (local address 127.11.192.193:44131)
0812 07:02:35.978761 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:35.978778 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:35.978927 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:35.979216 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:35.979221 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:35.979233 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:02:35.979567 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:35.979580 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:35.980792 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:35.980797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:35.981866 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:35.981879 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:35.982587 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:35.982594 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:35.982721 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:35.982757 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:02:35.983096 (+ 339us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:02:35.983107 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 07:02:35.983180 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 07:02:35.983311 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":322,"threads_started":1}
I20260812 07:02:36.012115 15010 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:02:36.012415 15010 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:02:36.015671 15010 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a: Sending DeleteTablet for 3 replicas of tablet d5fe90d6640c4a2daad1468f93024174
I20260812 07:02:36.016427 15256 tablet_service.cc:1514] Processing DeleteTablet for tablet d5fe90d6640c4a2daad1468f93024174 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:36 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:40660
I20260812 07:02:36.016867 16061 tablet_replica.cc:330] T d5fe90d6640c4a2daad1468f93024174 P 80bc5212b150451b93f08a4ff60cacc5: stopping tablet replica
I20260812 07:02:36.016918 15487 tablet_service.cc:1514] Processing DeleteTablet for tablet d5fe90d6640c4a2daad1468f93024174 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:36 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:39242
I20260812 07:02:36.017087 16061 raft_consensus.cc:2229] T d5fe90d6640c4a2daad1468f93024174 P 80bc5212b150451b93f08a4ff60cacc5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:02:36.017246 15742 tablet_service.cc:1514] Processing DeleteTablet for tablet d5fe90d6640c4a2daad1468f93024174 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:36 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:41620
I20260812 07:02:36.017279 16061 raft_consensus.cc:2258] T d5fe90d6640c4a2daad1468f93024174 P 80bc5212b150451b93f08a4ff60cacc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:36.017501 16064 tablet_replica.cc:330] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830: stopping tablet replica
I20260812 07:02:36.017511 16063 tablet_replica.cc:330] T d5fe90d6640c4a2daad1468f93024174 P 3cd38f56505e46dc8b8109aad1486cd4: stopping tablet replica
I20260812 07:02:36.017689 16063 raft_consensus.cc:2229] T d5fe90d6640c4a2daad1468f93024174 P 3cd38f56505e46dc8b8109aad1486cd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:02:36.017706 16064 raft_consensus.cc:2229] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:02:36.017892 16063 raft_consensus.cc:2258] T d5fe90d6640c4a2daad1468f93024174 P 3cd38f56505e46dc8b8109aad1486cd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:36.018184 16064 raft_consensus.cc:2258] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:36.018832 16061 ts_tablet_manager.cc:1905] T d5fe90d6640c4a2daad1468f93024174 P 80bc5212b150451b93f08a4ff60cacc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:36.019122 16063 ts_tablet_manager.cc:1905] T d5fe90d6640c4a2daad1468f93024174 P 3cd38f56505e46dc8b8109aad1486cd4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:36.019362 16064 ts_tablet_manager.cc:1905] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:36.024770 15009 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45910:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:02:36.025501 15009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:02:36.025866 16063 ts_tablet_manager.cc:1918] T d5fe90d6640c4a2daad1468f93024174 P 3cd38f56505e46dc8b8109aad1486cd4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:02:36.026007 16063 log.cc:1196] T d5fe90d6640c4a2daad1468f93024174 P 3cd38f56505e46dc8b8109aad1486cd4: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/d5fe90d6640c4a2daad1468f93024174
I20260812 07:02:36.026238 16064 ts_tablet_manager.cc:1918] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:02:36.026387 16063 ts_tablet_manager.cc:1939] T d5fe90d6640c4a2daad1468f93024174 P 3cd38f56505e46dc8b8109aad1486cd4: Deleting consensus metadata
I20260812 07:02:36.026397 16064 log.cc:1196] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/d5fe90d6640c4a2daad1468f93024174
I20260812 07:02:36.026959 16064 ts_tablet_manager.cc:1939] T d5fe90d6640c4a2daad1468f93024174 P 186f7554d2ce4a9a838664c036252830: Deleting consensus metadata
I20260812 07:02:36.027894 14980 catalog_manager.cc:4841] TS 3cd38f56505e46dc8b8109aad1486cd4 (127.11.192.194:34241): tablet d5fe90d6640c4a2daad1468f93024174 (table test-table [id=8b1a565167c04ea188cdd0b8fdd7e6fb]) successfully deleted
I20260812 07:02:36.028324 14980 catalog_manager.cc:4841] TS 186f7554d2ce4a9a838664c036252830 (127.11.192.195:36471): tablet d5fe90d6640c4a2daad1468f93024174 (table test-table [id=8b1a565167c04ea188cdd0b8fdd7e6fb]) successfully deleted
I20260812 07:02:36.029426 16061 ts_tablet_manager.cc:1918] T d5fe90d6640c4a2daad1468f93024174 P 80bc5212b150451b93f08a4ff60cacc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:02:36.029654 16061 log.cc:1196] T d5fe90d6640c4a2daad1468f93024174 P 80bc5212b150451b93f08a4ff60cacc5: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-0/wal/wals/d5fe90d6640c4a2daad1468f93024174
I20260812 07:02:36.030102 16061 ts_tablet_manager.cc:1939] T d5fe90d6640c4a2daad1468f93024174 P 80bc5212b150451b93f08a4ff60cacc5: Deleting consensus metadata
I20260812 07:02:36.031052 14980 catalog_manager.cc:4841] TS 80bc5212b150451b93f08a4ff60cacc5 (127.11.192.193:34835): tablet d5fe90d6640c4a2daad1468f93024174 (table test-table [id=8b1a565167c04ea188cdd0b8fdd7e6fb]) successfully deleted
I20260812 07:02:36.032801 15256 tablet_service.cc:1467] Processing CreateTablet for tablet cdeb06b126b54a6ab6071177ebf7a576 (DEFAULT_TABLE table=test-table [id=7b39e43f4d9344b5b5933dfba6adfcef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:36.033213 15256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdeb06b126b54a6ab6071177ebf7a576. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:36.033337 15742 tablet_service.cc:1467] Processing CreateTablet for tablet cdeb06b126b54a6ab6071177ebf7a576 (DEFAULT_TABLE table=test-table [id=7b39e43f4d9344b5b5933dfba6adfcef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:36.033739 15742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdeb06b126b54a6ab6071177ebf7a576. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:36.039067 16083 tablet_bootstrap.cc:492] T cdeb06b126b54a6ab6071177ebf7a576 P 80bc5212b150451b93f08a4ff60cacc5: Bootstrap starting.
I20260812 07:02:36.039882 16085 tablet_bootstrap.cc:492] T cdeb06b126b54a6ab6071177ebf7a576 P 186f7554d2ce4a9a838664c036252830: Bootstrap starting.
I20260812 07:02:36.040447 15487 tablet_service.cc:1467] Processing CreateTablet for tablet cdeb06b126b54a6ab6071177ebf7a576 (DEFAULT_TABLE table=test-table [id=7b39e43f4d9344b5b5933dfba6adfcef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:36.040747 16083 tablet_bootstrap.cc:654] T cdeb06b126b54a6ab6071177ebf7a576 P 80bc5212b150451b93f08a4ff60cacc5: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:36.040860 15487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdeb06b126b54a6ab6071177ebf7a576. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:36.041473 16085 tablet_bootstrap.cc:654] T cdeb06b126b54a6ab6071177ebf7a576 P 186f7554d2ce4a9a838664c036252830: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:36.047920 16086 tablet_bootstrap.cc:492] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4: Bootstrap starting.
I20260812 07:02:36.049456 16086 tablet_bootstrap.cc:654] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:36.051795 16086 tablet_bootstrap.cc:492] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4: No bootstrap required, opened a new log
I20260812 07:02:36.051954 16086 ts_tablet_manager.cc:1397] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 07:02:36.052521 16085 tablet_bootstrap.cc:492] T cdeb06b126b54a6ab6071177ebf7a576 P 186f7554d2ce4a9a838664c036252830: No bootstrap required, opened a new log
I20260812 07:02:36.052762 16085 ts_tablet_manager.cc:1397] T cdeb06b126b54a6ab6071177ebf7a576 P 186f7554d2ce4a9a838664c036252830: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.004s
I20260812 07:02:36.052762 16086 raft_consensus.cc:348] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:36.053007 16086 raft_consensus.cc:374] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:36.053093 16086 raft_consensus.cc:729] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cd38f56505e46dc8b8109aad1486cd4, State: Initialized, Role: FOLLOWER
I20260812 07:02:36.053368 16086 consensus_queue.cc:260] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:36.053539 16083 tablet_bootstrap.cc:492] T cdeb06b126b54a6ab6071177ebf7a576 P 80bc5212b150451b93f08a4ff60cacc5: No bootstrap required, opened a new log
I20260812 07:02:36.053699 16083 ts_tablet_manager.cc:1397] T cdeb06b126b54a6ab6071177ebf7a576 P 80bc5212b150451b93f08a4ff60cacc5: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.006s
I20260812 07:02:36.054456 16085 raft_consensus.cc:348] T cdeb06b126b54a6ab6071177ebf7a576 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:36.054795 16085 raft_consensus.cc:374] T cdeb06b126b54a6ab6071177ebf7a576 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:36.054945 16085 raft_consensus.cc:729] T cdeb06b126b54a6ab6071177ebf7a576 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 186f7554d2ce4a9a838664c036252830, State: Initialized, Role: FOLLOWER
I20260812 07:02:36.055012 16083 raft_consensus.cc:348] T cdeb06b126b54a6ab6071177ebf7a576 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:36.055279 16083 raft_consensus.cc:374] T cdeb06b126b54a6ab6071177ebf7a576 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:36.055377 16083 raft_consensus.cc:729] T cdeb06b126b54a6ab6071177ebf7a576 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80bc5212b150451b93f08a4ff60cacc5, State: Initialized, Role: FOLLOWER
I20260812 07:02:36.055332 16085 consensus_queue.cc:260] T cdeb06b126b54a6ab6071177ebf7a576 P 186f7554d2ce4a9a838664c036252830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:36.055622 16083 consensus_queue.cc:260] T cdeb06b126b54a6ab6071177ebf7a576 P 80bc5212b150451b93f08a4ff60cacc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:36.056160 16085 ts_tablet_manager.cc:1428] T cdeb06b126b54a6ab6071177ebf7a576 P 186f7554d2ce4a9a838664c036252830: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 07:02:36.056202 16083 ts_tablet_manager.cc:1428] T cdeb06b126b54a6ab6071177ebf7a576 P 80bc5212b150451b93f08a4ff60cacc5: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260812 07:02:36.056772 16086 ts_tablet_manager.cc:1428] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260812 07:02:36.216956 15933 raft_consensus.cc:482] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:02:36.217157 15933 raft_consensus.cc:504] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:36.217811 15933 leader_election.cc:288] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80bc5212b150451b93f08a4ff60cacc5 (127.11.192.193:34835), 186f7554d2ce4a9a838664c036252830 (127.11.192.195:36471)
I20260812 07:02:36.218493 15306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdeb06b126b54a6ab6071177ebf7a576" candidate_uuid: "3cd38f56505e46dc8b8109aad1486cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80bc5212b150451b93f08a4ff60cacc5" is_pre_election: true
I20260812 07:02:36.218544 15779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdeb06b126b54a6ab6071177ebf7a576" candidate_uuid: "3cd38f56505e46dc8b8109aad1486cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "186f7554d2ce4a9a838664c036252830" is_pre_election: true
I20260812 07:02:36.218740 15306 raft_consensus.cc:2454] T cdeb06b126b54a6ab6071177ebf7a576 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3cd38f56505e46dc8b8109aad1486cd4 in term 0.
I20260812 07:02:36.218869 15779 raft_consensus.cc:2454] T cdeb06b126b54a6ab6071177ebf7a576 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3cd38f56505e46dc8b8109aad1486cd4 in term 0.
I20260812 07:02:36.219236 15409 leader_election.cc:302] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cd38f56505e46dc8b8109aad1486cd4, 80bc5212b150451b93f08a4ff60cacc5; no voters:
I20260812 07:02:36.219519 15933 raft_consensus.cc:2781] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:02:36.219647 15933 raft_consensus.cc:482] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:02:36.219740 15933 raft_consensus.cc:3037] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:36.221709 15933 raft_consensus.cc:504] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:36.222208 15933 leader_election.cc:288] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4 [CANDIDATE]: Term 1 election: Requested vote from peers 80bc5212b150451b93f08a4ff60cacc5 (127.11.192.193:34835), 186f7554d2ce4a9a838664c036252830 (127.11.192.195:36471)
I20260812 07:02:36.222684 15306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdeb06b126b54a6ab6071177ebf7a576" candidate_uuid: "3cd38f56505e46dc8b8109aad1486cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80bc5212b150451b93f08a4ff60cacc5"
I20260812 07:02:36.222915 15306 raft_consensus.cc:3037] T cdeb06b126b54a6ab6071177ebf7a576 P 80bc5212b150451b93f08a4ff60cacc5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:36.222967 15779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdeb06b126b54a6ab6071177ebf7a576" candidate_uuid: "3cd38f56505e46dc8b8109aad1486cd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "186f7554d2ce4a9a838664c036252830"
I20260812 07:02:36.223181 15779 raft_consensus.cc:3037] T cdeb06b126b54a6ab6071177ebf7a576 P 186f7554d2ce4a9a838664c036252830 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:36.225503 15779 raft_consensus.cc:2454] T cdeb06b126b54a6ab6071177ebf7a576 P 186f7554d2ce4a9a838664c036252830 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cd38f56505e46dc8b8109aad1486cd4 in term 1.
I20260812 07:02:36.225894 15409 leader_election.cc:302] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 186f7554d2ce4a9a838664c036252830, 3cd38f56505e46dc8b8109aad1486cd4; no voters:
I20260812 07:02:36.225903 15306 raft_consensus.cc:2454] T cdeb06b126b54a6ab6071177ebf7a576 P 80bc5212b150451b93f08a4ff60cacc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cd38f56505e46dc8b8109aad1486cd4 in term 1.
I20260812 07:02:36.226167 15933 raft_consensus.cc:2781] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:02:36.226965 15933 raft_consensus.cc:686] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4 [term 1 LEADER]: Becoming Leader. State: Replica: 3cd38f56505e46dc8b8109aad1486cd4, State: Running, Role: LEADER
I20260812 07:02:36.227566 15933 consensus_queue.cc:237] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } }
I20260812 07:02:36.230050 15010 catalog_manager.cc:5495] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3cd38f56505e46dc8b8109aad1486cd4 (127.11.192.194). New cstate: current_term: 1 leader_uuid: "3cd38f56505e46dc8b8109aad1486cd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3cd38f56505e46dc8b8109aad1486cd4" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 } health_report { overall_health: UNKNOWN } } }
I20260812 07:02:36.292806 16102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:36.288466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39262 (local address 127.11.192.194:34241)
0812 07:02:36.288638 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:36.288643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:36.288661 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:02:36.288722 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:36.288727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:36.288798 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:36.288874 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:36.289243 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:36.289957 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:36.291229 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:36.291634 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:36.291742 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:36.292167 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:36.292273 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:36.292360 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:36.292499 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 07:02:36.292572 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:02:36.301965 15306 raft_consensus.cc:1261] T cdeb06b126b54a6ab6071177ebf7a576 P 80bc5212b150451b93f08a4ff60cacc5 [term 1 FOLLOWER]: Refusing update from remote peer 3cd38f56505e46dc8b8109aad1486cd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:36.302668 16141 consensus_queue.cc:1035] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80bc5212b150451b93f08a4ff60cacc5" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:02:36.302932 15779 raft_consensus.cc:1261] T cdeb06b126b54a6ab6071177ebf7a576 P 186f7554d2ce4a9a838664c036252830 [term 1 FOLLOWER]: Refusing update from remote peer 3cd38f56505e46dc8b8109aad1486cd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:36.303627 16141 consensus_queue.cc:1035] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "186f7554d2ce4a9a838664c036252830" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 36471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 07:02:36.330050 16035 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 07:02:37.916635 15010 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:02:37.916838 15010 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:02:37.920540 15010 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce5f6fcb794a45798500c9063f62535a: Sending DeleteTablet for 3 replicas of tablet cdeb06b126b54a6ab6071177ebf7a576
I20260812 07:02:37.921317 15487 tablet_service.cc:1514] Processing DeleteTablet for tablet cdeb06b126b54a6ab6071177ebf7a576 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:37 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:39242
I20260812 07:02:37.921327 15256 tablet_service.cc:1514] Processing DeleteTablet for tablet cdeb06b126b54a6ab6071177ebf7a576 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:37 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:40660
I20260812 07:02:37.921363 15742 tablet_service.cc:1514] Processing DeleteTablet for tablet cdeb06b126b54a6ab6071177ebf7a576 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:37 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:41620
I20260812 07:02:37.921906 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 15143
I20260812 07:02:37.934111 16194 tablet_replica.cc:330] T cdeb06b126b54a6ab6071177ebf7a576 P 80bc5212b150451b93f08a4ff60cacc5: stopping tablet replica
I20260812 07:02:37.934300 16194 raft_consensus.cc:2229] T cdeb06b126b54a6ab6071177ebf7a576 P 80bc5212b150451b93f08a4ff60cacc5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:02:37.934624 16195 tablet_replica.cc:330] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4: stopping tablet replica
I20260812 07:02:37.934782 16196 tablet_replica.cc:330] T cdeb06b126b54a6ab6071177ebf7a576 P 186f7554d2ce4a9a838664c036252830: stopping tablet replica
I20260812 07:02:37.934787 16194 raft_consensus.cc:2258] T cdeb06b126b54a6ab6071177ebf7a576 P 80bc5212b150451b93f08a4ff60cacc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:37.934813 16195 raft_consensus.cc:2229] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:02:37.935016 16196 raft_consensus.cc:2229] T cdeb06b126b54a6ab6071177ebf7a576 P 186f7554d2ce4a9a838664c036252830 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:02:37.935279 16196 raft_consensus.cc:2258] T cdeb06b126b54a6ab6071177ebf7a576 P 186f7554d2ce4a9a838664c036252830 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:37.935266 16195 raft_consensus.cc:2258] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:37.935902 16194 ts_tablet_manager.cc:1905] T cdeb06b126b54a6ab6071177ebf7a576 P 80bc5212b150451b93f08a4ff60cacc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:37.936157 16195 ts_tablet_manager.cc:1905] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:37.936748 16196 ts_tablet_manager.cc:1905] T cdeb06b126b54a6ab6071177ebf7a576 P 186f7554d2ce4a9a838664c036252830: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:37.942443 16196 ts_tablet_manager.cc:1918] T cdeb06b126b54a6ab6071177ebf7a576 P 186f7554d2ce4a9a838664c036252830: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 07:02:37.942574 16196 log.cc:1196] T cdeb06b126b54a6ab6071177ebf7a576 P 186f7554d2ce4a9a838664c036252830: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/cdeb06b126b54a6ab6071177ebf7a576
I20260812 07:02:37.942760 16194 ts_tablet_manager.cc:1918] T cdeb06b126b54a6ab6071177ebf7a576 P 80bc5212b150451b93f08a4ff60cacc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 07:02:37.942744 16195 ts_tablet_manager.cc:1918] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 07:02:37.942889 16195 log.cc:1196] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/cdeb06b126b54a6ab6071177ebf7a576
I20260812 07:02:37.942889 16194 log.cc:1196] T cdeb06b126b54a6ab6071177ebf7a576 P 80bc5212b150451b93f08a4ff60cacc5: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518136653111-12035-0/minicluster-data/ts-0/wal/wals/cdeb06b126b54a6ab6071177ebf7a576
I20260812 07:02:37.942974 16196 ts_tablet_manager.cc:1939] T cdeb06b126b54a6ab6071177ebf7a576 P 186f7554d2ce4a9a838664c036252830: Deleting consensus metadata
I20260812 07:02:37.943333 16194 ts_tablet_manager.cc:1939] T cdeb06b126b54a6ab6071177ebf7a576 P 80bc5212b150451b93f08a4ff60cacc5: Deleting consensus metadata
I20260812 07:02:37.943333 16195 ts_tablet_manager.cc:1939] T cdeb06b126b54a6ab6071177ebf7a576 P 3cd38f56505e46dc8b8109aad1486cd4: Deleting consensus metadata
I20260812 07:02:37.943907 14980 catalog_manager.cc:4841] TS 186f7554d2ce4a9a838664c036252830 (127.11.192.195:36471): tablet cdeb06b126b54a6ab6071177ebf7a576 (table test-table [id=7b39e43f4d9344b5b5933dfba6adfcef]) successfully deleted
I20260812 07:02:37.944331 14980 catalog_manager.cc:4841] TS 80bc5212b150451b93f08a4ff60cacc5 (127.11.192.193:34835): tablet cdeb06b126b54a6ab6071177ebf7a576 (table test-table [id=7b39e43f4d9344b5b5933dfba6adfcef]) successfully deleted
I20260812 07:02:37.944537 14980 catalog_manager.cc:4841] TS 3cd38f56505e46dc8b8109aad1486cd4 (127.11.192.194:34241): tablet cdeb06b126b54a6ab6071177ebf7a576 (table test-table [id=7b39e43f4d9344b5b5933dfba6adfcef]) successfully deleted
Aug 12 07:02:37 dist-test-slave-lgr5 krb5kdc[14931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518155, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.192.193@KRBTEST.COM
I20260812 07:02:37.954528 16199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:37.923427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40682 (local address 127.11.192.193:34835)
0812 07:02:37.934174 (+ 10747us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:37.934180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:37.934204 (+ 24us) server_negotiation.cc:407] Connection header received
0812 07:02:37.934262 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:37.934268 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:37.934346 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:37.936188 (+ 1842us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:37.936412 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:37.937609 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:37.939446 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:37.940118 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:37.941186 (+ 1068us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:37.948886 (+ 7700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:37.948913 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:37.948936 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:37.948988 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:37.951845 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:37.952537 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:37.952545 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:37.952547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:37.952683 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:37.953039 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:37.953043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:37.953045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:37.953382 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:37.953677 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:37.953957 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 07:02:37.954170 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10612,"thread_start_us":10432,"threads_started":1}
W20260812 07:02:37.955127 15371 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40682
W20260812 07:02:37.955670 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 15143: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:02:37.955868 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 15143
I20260812 07:02:37.986405 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 15382
Aug 12 07:02:37 dist-test-slave-lgr5 krb5kdc[14931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518155, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.192.194@KRBTEST.COM
I20260812 07:02:38.000259 16203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:37.987103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39266 (local address 127.11.192.194:34241)
0812 07:02:37.987606 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:37.987612 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:37.987627 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:02:37.987681 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:37.987684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:37.987740 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:37.987825 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:37.989052 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:37.989790 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:37.990935 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:37.991318 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:37.991419 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:37.995229 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:37.995243 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:37.995259 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:37.995298 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:37.998055 (+ 2757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:37.998593 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:37.998599 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:37.998601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:37.998730 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:37.999011 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:37.999014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:37.999016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:37.999324 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:37.999553 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:37.999731 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 07:02:37.999885 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":243,"threads_started":1}
W20260812 07:02:38.000880 15588 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39266
W20260812 07:02:38.001473 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 15382: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:02:38.001605 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 15382
I20260812 07:02:38.034183 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 15608
Aug 12 07:02:38 dist-test-slave-lgr5 krb5kdc[14931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518155, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.192.195@KRBTEST.COM
I20260812 07:02:38.048070 16205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:38.034866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41642 (local address 127.11.192.195:36471)
0812 07:02:38.035329 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:38.035334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:38.035350 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:02:38.035404 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:38.035408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:38.035474 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:38.035565 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:38.036775 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:38.037573 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:38.038777 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:38.039188 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:38.039278 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:38.043136 (+ 3858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:38.043151 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:38.043167 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:38.043204 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:38.045868 (+ 2664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:38.046407 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:38.046414 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:38.046417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:38.046574 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:38.046838 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:38.046840 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:38.046842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:38.047190 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:38.047411 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:38.047590 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 07:02:38.047744 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":245,"threads_started":1}
W20260812 07:02:38.048715 15867 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41642
W20260812 07:02:38.049289 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 15608: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:02:38.049433 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 15608
I20260812 07:02:38.075177 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 14955
I20260812 07:02:38.076314 15081 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:38.194803 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 14955
2026-08-12T07:02:38Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8728 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:02:38 dist-test-slave-lgr5 krb5kdc[16214](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:02:38 dist-test-slave-lgr5 krb5kdc[16214](info): set up 2 sockets
Aug 12 07:02:38 dist-test-slave-lgr5 krb5kdc[16214](info): commencing operation
krb5kdc: starting...
W20260812 07:02:39.423651 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:02:39 dist-test-slave-lgr5 krb5kdc[16214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:02:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:02:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:02:39.600387 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:41551
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:41551
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:39.906172 16232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:39.906450 16232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:39.906570 16232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:39.913482 16232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:02:39.913573 16232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:39.913625 16232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:39.913678 16232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:02:39.913722 16232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:02:39.922906 16232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:41551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:41551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:39.923966 16232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:39.925153 16232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:39.936161 16246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:39.936208 16241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:39.936239 16243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:39.936865 16232 server_base.cc:1034] running on GCE node
I20260812 07:02:39.937623 16232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:39.938890 16232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:39.940117 16232 hybrid_clock.cc:648] HybridClock initialized: now 1786518159940037 us; error 46 us; skew 500 ppm
I20260812 07:02:39.940454 16232 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:39 dist-test-slave-lgr5 krb5kdc[16214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518159, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:39.943369 16232 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:02:39.944715 16232 webserver.cc:460] Webserver started at http://127.11.192.254:41227/ using document root <none> and password file <none>
I20260812 07:02:39.945333 16232 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:39.945428 16232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:39.945676 16232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:39.948454 16232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "822a45e3879241d9a88bc8ec8a24a4a3"
format_stamp: "Formatted at 2026-08-12 07:02:39 on dist-test-slave-lgr5"
server_key: "aedb69d92fae0bbb7256f630c26829ce"
server_key_iv: "2761568ec4d79a8ce59273e5704c8515"
server_key_version: "encryptionkey@0"
I20260812 07:02:39.949162 16232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "822a45e3879241d9a88bc8ec8a24a4a3"
format_stamp: "Formatted at 2026-08-12 07:02:39 on dist-test-slave-lgr5"
server_key: "aedb69d92fae0bbb7256f630c26829ce"
server_key_iv: "2761568ec4d79a8ce59273e5704c8515"
server_key_version: "encryptionkey@0"
I20260812 07:02:39.953876 16232 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:02:39.957057 16262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:39.958278 16232 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:02:39.958437 16232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "822a45e3879241d9a88bc8ec8a24a4a3"
format_stamp: "Formatted at 2026-08-12 07:02:39 on dist-test-slave-lgr5"
server_key: "aedb69d92fae0bbb7256f630c26829ce"
server_key_iv: "2761568ec4d79a8ce59273e5704c8515"
server_key_version: "encryptionkey@0"
I20260812 07:02:39.958606 16232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:39.975126 16232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:39.983178 16232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:39.985314 16232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:40.011485 16232 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:41551
I20260812 07:02:40.011515 16347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:41551 every 8 connection(s)
I20260812 07:02:40.013130 16232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:02:40.016880 16350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:40.017405 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 16232
I20260812 07:02:40.017611 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:02:40.018229 12035 external_mini_cluster.cc:1428] Setting key 84f143f305842191587cdc1ae84203e4
I20260812 07:02:40.028548 16350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3: Bootstrap starting.
Aug 12 07:02:40 dist-test-slave-lgr5 krb5kdc[16214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:40.031639 16350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:40.032990 16350 log.cc:824] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:40.035786 16350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3: No bootstrap required, opened a new log
I20260812 07:02:40.039875 16354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:40.020176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54212 (local address 127.11.192.254:41551)
0812 07:02:40.020909 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:40.020921 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:40.020963 (+ 42us) server_negotiation.cc:407] Connection header received
0812 07:02:40.022689 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:40.022711 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:40.023081 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:40.023427 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:40.024904 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:40.026210 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:40.027487 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:40.028213 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:40.028378 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:40.032416 (+ 4038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:40.032424 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:40.032437 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:40.032470 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:40.035569 (+ 3099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:40.036309 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:40.036317 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:40.036325 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:40.036540 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:40.036872 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:40.036876 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:40.036879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:40.037566 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:40.037871 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:40.038186 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 07:02:40.038610 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":297,"threads_started":1}
I20260812 07:02:40.044772 16350 raft_consensus.cc:348] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822a45e3879241d9a88bc8ec8a24a4a3" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 41551 } }
I20260812 07:02:40.045070 16350 raft_consensus.cc:374] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:40.045166 16350 raft_consensus.cc:729] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 822a45e3879241d9a88bc8ec8a24a4a3, State: Initialized, Role: FOLLOWER
I20260812 07:02:40.045706 16350 consensus_queue.cc:260] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822a45e3879241d9a88bc8ec8a24a4a3" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 41551 } }
I20260812 07:02:40.045917 16350 raft_consensus.cc:388] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:02:40.046041 16350 raft_consensus.cc:482] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:02:40.046195 16350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:40.048185 16350 raft_consensus.cc:504] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822a45e3879241d9a88bc8ec8a24a4a3" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 41551 } }
I20260812 07:02:40.048631 16350 leader_election.cc:302] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 822a45e3879241d9a88bc8ec8a24a4a3; no voters:
I20260812 07:02:40.049124 16350 leader_election.cc:288] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:02:40.049252 16359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:02:40.050431 16359 raft_consensus.cc:686] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3 [term 1 LEADER]: Becoming Leader. State: Replica: 822a45e3879241d9a88bc8ec8a24a4a3, State: Running, Role: LEADER
I20260812 07:02:40.050879 16359 consensus_queue.cc:237] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822a45e3879241d9a88bc8ec8a24a4a3" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 41551 } }
I20260812 07:02:40.051174 16350 sys_catalog.cc:564] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:02:40.053236 16362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 822a45e3879241d9a88bc8ec8a24a4a3. Latest consensus state: current_term: 1 leader_uuid: "822a45e3879241d9a88bc8ec8a24a4a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822a45e3879241d9a88bc8ec8a24a4a3" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 41551 } } }
I20260812 07:02:40.053596 16362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3 [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:40.054523 16361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "822a45e3879241d9a88bc8ec8a24a4a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822a45e3879241d9a88bc8ec8a24a4a3" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 41551 } } }
I20260812 07:02:40.054970 16361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3 [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:40.055564 16371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:02:40.065191 16371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:02:40.074044 16371 catalog_manager.cc:1347] Generated new cluster ID: 71db1c31d3a349f39e213ac84a1dbdc1
I20260812 07:02:40.074160 16371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:02:40.093130 16371 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:02:40.094913 16371 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:02:40.109496 16371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 822a45e3879241d9a88bc8ec8a24a4a3: Generated new TSK 0
I20260812 07:02:40.110476 16371 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
I20260812 07:02:40.167213 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:41551
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:40.481155 16405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:40.481427 16405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:40.481571 16405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:40.488343 16405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:40.488435 16405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:40.488585 16405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:02:40.497713 16405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:41551
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:40.498826 16405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:40.500025 16405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:40.512162 16423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:40.512238 16415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:40.512213 16419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:40.513068 16405 server_base.cc:1034] running on GCE node
I20260812 07:02:40.513612 16405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:40.514434 16405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:40.515632 16405 hybrid_clock.cc:648] HybridClock initialized: now 1786518160515590 us; error 36 us; skew 500 ppm
I20260812 07:02:40.515910 16405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:40 dist-test-slave-lgr5 krb5kdc[16214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518160, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:40.518863 16405 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:02:40.520228 16405 webserver.cc:460] Webserver started at http://127.11.192.193:35573/ using document root <none> and password file <none>
I20260812 07:02:40.520850 16405 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:40.520963 16405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:40.521251 16405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:40.523923 16405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "72b144ba4c5045c399b10afc2def9c4b"
format_stamp: "Formatted at 2026-08-12 07:02:40 on dist-test-slave-lgr5"
server_key: "6294d8507b5eb2f49783c71ac68f6e43"
server_key_iv: "f8202807f11dd7d50809af6e443253c4"
server_key_version: "encryptionkey@0"
I20260812 07:02:40.524710 16405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "72b144ba4c5045c399b10afc2def9c4b"
format_stamp: "Formatted at 2026-08-12 07:02:40 on dist-test-slave-lgr5"
server_key: "6294d8507b5eb2f49783c71ac68f6e43"
server_key_iv: "f8202807f11dd7d50809af6e443253c4"
server_key_version: "encryptionkey@0"
I20260812 07:02:40.529351 16405 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 07:02:40.532536 16432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:40.533713 16405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 07:02:40.533866 16405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "72b144ba4c5045c399b10afc2def9c4b"
format_stamp: "Formatted at 2026-08-12 07:02:40 on dist-test-slave-lgr5"
server_key: "6294d8507b5eb2f49783c71ac68f6e43"
server_key_iv: "f8202807f11dd7d50809af6e443253c4"
server_key_version: "encryptionkey@0"
I20260812 07:02:40.534049 16405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:40.554587 16405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:40.558502 16405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:40.558753 16405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:40.559618 16405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:02:40.560889 16405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:40.560972 16405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:40.561087 16405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:40.561141 16405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:40.590740 16405 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:45995
I20260812 07:02:40.590797 16625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:45995 every 8 connection(s)
I20260812 07:02:40.591929 16405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:02:40.594511 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 16405
I20260812 07:02:40.594731 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:02:40.595575 12035 external_mini_cluster.cc:1428] Setting key 48bef27a517498debda9ed30eca54469
Aug 12 07:02:40 dist-test-slave-lgr5 krb5kdc[16214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518160, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:40.610986 16629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:40.594239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:53869 (local address 127.11.192.254:41551)
0812 07:02:40.594651 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:40.594655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:40.595811 (+ 1156us) server_negotiation.cc:407] Connection header received
0812 07:02:40.598059 (+ 2248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:40.598064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:40.598141 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:40.598232 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:40.599664 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:40.600450 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:40.601921 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:40.602322 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:40.602430 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:40.605731 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:40.605736 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:40.605739 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:40.605770 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:40.608125 (+ 2355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:40.608804 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:40.608810 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:40.608813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:40.608931 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:40.609355 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:40.609360 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:40.609363 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:40.609710 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:40.609852 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:40.610566 (+ 714us) server_negotiation.cc:299] Negotiation successful
0812 07:02:40.610728 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":211,"threads_started":1}
I20260812 07:02:40.612286 16637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:40.594894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:41551 (local address 127.11.192.193:53869)
0812 07:02:40.595664 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:40.595712 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:40.597819 (+ 2107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:40.598467 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:40.598474 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:40.598975 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:40.599501 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:40.599519 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:40.600603 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:40.600607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:40.601784 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:40.601797 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:40.602535 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:40.602543 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:40.602770 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:40.603370 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:40.603392 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:40.605522 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:40.608248 (+ 2726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:40.608256 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:40.608270 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:40.608691 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:40.609056 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:40.609061 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:40.609065 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:40.609245 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:40.609862 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:40.609870 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:40.610231 (+ 361us) client_negotiation.cc:769] Sending connection context
0812 07:02:40.610516 (+ 285us) client_negotiation.cc:240] Negotiation successful
0812 07:02:40.610980 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":297,"threads_started":1}
I20260812 07:02:40.613709 16626 heartbeater.cc:347] Connected to a master server at 127.11.192.254:41551
I20260812 07:02:40.614015 16626 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:40.614840 16626 heartbeater.cc:511] Master 127.11.192.254:41551 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.192.194@KRBTEST.COM; defaulting to no policy
I20260812 07:02:40.617063 16289 ts_manager.cc:194] Registered new tserver with Master: 72b144ba4c5045c399b10afc2def9c4b (127.11.192.193:45995)
I20260812 07:02:40.618501 16289 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:53869
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
I20260812 07:02:40.639964 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:41551
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:40.946235 16644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:40.946484 16644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:40.946617 16644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:40.953544 16644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:40.953650 16644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:40.953791 16644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:02:40.962826 16644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:41551
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:40.963961 16644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:40.965262 16644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:40.977442 16652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:40.977495 16654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:40.977514 16656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:40.978475 16644 server_base.cc:1034] running on GCE node
I20260812 07:02:40.979031 16644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:40.979876 16644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:40.981066 16644 hybrid_clock.cc:648] HybridClock initialized: now 1786518160981006 us; error 47 us; skew 500 ppm
I20260812 07:02:40.981350 16644 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:40 dist-test-slave-lgr5 krb5kdc[16214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518160, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:40.984434 16644 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:02:40.985678 16644 webserver.cc:460] Webserver started at http://127.11.192.194:40357/ using document root <none> and password file <none>
I20260812 07:02:40.986266 16644 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:40.986361 16644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:40.986603 16644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:40.989347 16644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "a230570aa79240fe97fbd19db93d1ef4"
format_stamp: "Formatted at 2026-08-12 07:02:40 on dist-test-slave-lgr5"
server_key: "ba7f2ea6ba138244126c05dd405a4d02"
server_key_iv: "42eab98c8cf2fd39ab751a919441f153"
server_key_version: "encryptionkey@0"
I20260812 07:02:40.990057 16644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "a230570aa79240fe97fbd19db93d1ef4"
format_stamp: "Formatted at 2026-08-12 07:02:40 on dist-test-slave-lgr5"
server_key: "ba7f2ea6ba138244126c05dd405a4d02"
server_key_iv: "42eab98c8cf2fd39ab751a919441f153"
server_key_version: "encryptionkey@0"
I20260812 07:02:40.994728 16644 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:02:40.997889 16665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:40.999061 16644 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:02:40.999189 16644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "a230570aa79240fe97fbd19db93d1ef4"
format_stamp: "Formatted at 2026-08-12 07:02:40 on dist-test-slave-lgr5"
server_key: "ba7f2ea6ba138244126c05dd405a4d02"
server_key_iv: "42eab98c8cf2fd39ab751a919441f153"
server_key_version: "encryptionkey@0"
I20260812 07:02:40.999346 16644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:41.041193 16644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:41.044948 16644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:41.045199 16644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:41.046070 16644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:02:41.047317 16644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:41.047390 16644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:41.047478 16644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:41.047520 16644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:41.076623 16644 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:45155
I20260812 07:02:41.076671 16868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:45155 every 8 connection(s)
I20260812 07:02:41.077826 16644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:02:41.084705 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 16644
I20260812 07:02:41.084908 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:02:41.085456 12035 external_mini_cluster.cc:1428] Setting key 9055048c9039a86e38462ff76a706728
Aug 12 07:02:41 dist-test-slave-lgr5 krb5kdc[16214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518160, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:41.098027 16629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:41.080188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:35579 (local address 127.11.192.254:41551)
0812 07:02:41.080351 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:41.080356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:41.081464 (+ 1108us) server_negotiation.cc:407] Connection header received
0812 07:02:41.083341 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:41.083346 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:41.083428 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:41.083524 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:41.085326 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:41.086198 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:41.088031 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:41.088491 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:41.088601 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:41.092526 (+ 3925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:41.092531 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:41.092534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:41.092566 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:41.095006 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:41.095786 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:41.095791 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:41.095795 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:41.095951 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:41.096371 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:41.096376 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:41.096378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:41.096713 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:41.096867 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:41.097460 (+ 593us) server_negotiation.cc:299] Negotiation successful
0812 07:02:41.097680 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:02:41.098698 16872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:41.080647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:41551 (local address 127.11.192.194:35579)
0812 07:02:41.081310 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:41.081346 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:41.083080 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:41.083781 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:41.083790 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:41.084328 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:41.085104 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:41.085131 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:41.086370 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:41.086375 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:41.087878 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:41.087894 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:41.088713 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:41.088720 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:41.088994 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:41.089704 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:41.089732 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:41.092305 (+ 2573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:41.095149 (+ 2844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:41.095157 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:41.095171 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:41.095657 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:41.096056 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:41.096059 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:41.096061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:41.096262 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:41.096820 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:41.096825 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:41.097118 (+ 293us) client_negotiation.cc:769] Sending connection context
0812 07:02:41.097325 (+ 207us) client_negotiation.cc:240] Negotiation successful
0812 07:02:41.097680 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":264,"threads_started":1}
I20260812 07:02:41.100015 16869 heartbeater.cc:347] Connected to a master server at 127.11.192.254:41551
I20260812 07:02:41.100313 16869 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:41.100984 16869 heartbeater.cc:511] Master 127.11.192.254:41551 requested a full tablet report, sending...
I20260812 07:02:41.102352 16289 ts_manager.cc:194] Registered new tserver with Master: a230570aa79240fe97fbd19db93d1ef4 (127.11.192.194:45155)
I20260812 07:02:41.103255 16289 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:35579
WARNING: no policy specified for kudu/127.11.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
I20260812 07:02:41.130441 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:41551
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:41.441707 16879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:41.441941 16879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:41.442036 16879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:41.449561 16879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:41.449651 16879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:41.449760 16879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:02:41.458813 16879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:41551
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:41.459890 16879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:41.461056 16879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:41.473420 16889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:41.473465 16895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:41.473498 16888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:41.474241 16879 server_base.cc:1034] running on GCE node
I20260812 07:02:41.474797 16879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:41.475632 16879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:41.476827 16879 hybrid_clock.cc:648] HybridClock initialized: now 1786518161476772 us; error 46 us; skew 500 ppm
I20260812 07:02:41.477097 16879 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:41 dist-test-slave-lgr5 krb5kdc[16214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518161, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:41.479957 16879 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:02:41.481303 16879 webserver.cc:460] Webserver started at http://127.11.192.195:46729/ using document root <none> and password file <none>
I20260812 07:02:41.481913 16879 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:41.482017 16879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:41.482301 16879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:41.485200 16879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "6fb8c8dae29747469960068084c7f560"
format_stamp: "Formatted at 2026-08-12 07:02:41 on dist-test-slave-lgr5"
server_key: "d496634dead36e5f8635d9c5edb7df96"
server_key_iv: "4c3c1da0032ba4f6418caaab7eeb4861"
server_key_version: "encryptionkey@0"
I20260812 07:02:41.485944 16879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "6fb8c8dae29747469960068084c7f560"
format_stamp: "Formatted at 2026-08-12 07:02:41 on dist-test-slave-lgr5"
server_key: "d496634dead36e5f8635d9c5edb7df96"
server_key_iv: "4c3c1da0032ba4f6418caaab7eeb4861"
server_key_version: "encryptionkey@0"
I20260812 07:02:41.490692 16879 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:02:41.493835 16909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:41.495016 16879 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:02:41.495168 16879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "6fb8c8dae29747469960068084c7f560"
format_stamp: "Formatted at 2026-08-12 07:02:41 on dist-test-slave-lgr5"
server_key: "d496634dead36e5f8635d9c5edb7df96"
server_key_iv: "4c3c1da0032ba4f6418caaab7eeb4861"
server_key_version: "encryptionkey@0"
I20260812 07:02:41.495350 16879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:41.519876 16879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:41.523347 16879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:41.523588 16879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:41.524449 16879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:02:41.525667 16879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:41.525741 16879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:41.525857 16879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:41.525906 16879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:41.555279 16879 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:35707
I20260812 07:02:41.555342 17109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:35707 every 8 connection(s)
I20260812 07:02:41.556452 16879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:02:41.566236 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 16879
I20260812 07:02:41.566406 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:02:41.566813 12035 external_mini_cluster.cc:1428] Setting key febc4967c0f94475ac1ff3efc79df5bc
Aug 12 07:02:41 dist-test-slave-lgr5 krb5kdc[16214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518161, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:41.574517 16629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:41.558690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:51549 (local address 127.11.192.254:41551)
0812 07:02:41.558907 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:41.558913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:41.559969 (+ 1056us) server_negotiation.cc:407] Connection header received
0812 07:02:41.561743 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:41.561751 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:41.561838 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:41.561925 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:41.563386 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:41.564162 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:41.565705 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:41.566165 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:41.566276 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:41.569827 (+ 3551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:41.569831 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:41.569835 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:41.569865 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:41.572141 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:41.572743 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:41.572746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:41.572748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:41.572869 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:41.573232 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:41.573235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:41.573237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:41.573481 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:41.573584 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:41.574123 (+ 539us) server_negotiation.cc:299] Negotiation successful
0812 07:02:41.574283 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:02:41.575350 17112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:41.559158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:41551 (local address 127.11.192.195:51549)
0812 07:02:41.559817 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:41.559853 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:41.561491 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:41.562138 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:41.562149 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:41.562669 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:41.563237 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:41.563262 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:41.564292 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:41.564298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:41.565569 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:41.565587 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:41.566390 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:41.566399 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:41.566631 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:41.567246 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:41.567278 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:41.569631 (+ 2353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:41.572253 (+ 2622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:41.572260 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:41.572269 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:41.572652 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:41.572963 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:41.572975 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:41.572978 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:41.573153 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:41.573570 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:41.573575 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:41.573836 (+ 261us) client_negotiation.cc:769] Sending connection context
0812 07:02:41.574060 (+ 224us) client_negotiation.cc:240] Negotiation successful
0812 07:02:41.574390 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":256,"threads_started":1}
I20260812 07:02:41.576478 17110 heartbeater.cc:347] Connected to a master server at 127.11.192.254:41551
I20260812 07:02:41.576740 17110 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:41.577384 17110 heartbeater.cc:511] Master 127.11.192.254:41551 requested a full tablet report, sending...
I20260812 07:02:41.578619 16289 ts_manager.cc:194] Registered new tserver with Master: 6fb8c8dae29747469960068084c7f560 (127.11.192.195:35707)
I20260812 07:02:41.579416 16289 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:51549
I20260812 07:02:41.581684 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 07:02:41.614810 16629 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:02:41.612505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54218 (local address 127.11.192.254:41551)
0812 07:02:41.612668 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:41.612674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:41.613100 (+ 426us) server_negotiation.cc:407] Connection header received
0812 07:02:41.613264 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:41.613268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:41.613338 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:41.613421 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:41.614576 (+ 1155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54218: BlockingRecv error: recv got EOF from 127.0.0.1:54218 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:02:41.617776 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 16405
W20260812 07:02:41.620198 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 16405: Not authorized: Client connection negotiation failed: client connection to 127.11.192.193:45995: server requires authentication, but client does not have Kerberos credentials available
I20260812 07:02:41.620374 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 16405
I20260812 07:02:41.642773 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 16644
W20260812 07:02:41.645092 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 16644: Not authorized: Client connection negotiation failed: client connection to 127.11.192.194:45155: server requires authentication, but client does not have Kerberos credentials available
I20260812 07:02:41.645216 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 16644
I20260812 07:02:41.666553 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 16879
W20260812 07:02:41.669229 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 16879: Not authorized: Client connection negotiation failed: client connection to 127.11.192.195:35707: server requires authentication, but client does not have Kerberos credentials available
I20260812 07:02:41.669345 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 16879
I20260812 07:02:41.687939 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 16232
I20260812 07:02:41.689042 16343 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:41.789011 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 16232
2026-08-12T07:02:41Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3597 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:02:41 dist-test-slave-lgr5 krb5kdc[17135](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:02:41 dist-test-slave-lgr5 krb5kdc[17135](info): set up 2 sockets
Aug 12 07:02:41 dist-test-slave-lgr5 krb5kdc[17135](info): commencing operation
krb5kdc: starting...
W20260812 07:02:43.023149 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.001s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:02:43 dist-test-slave-lgr5 krb5kdc[17135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:02:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:02:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:02:43.207269 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:39107
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:39107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:43.515657 17165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:43.515923 17165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:43.516067 17165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:43.523046 17165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:02:43.523149 17165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:43.523238 17165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:43.523332 17165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:02:43.523411 17165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:02:43.532856 17165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:39107
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:39107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:43.534003 17165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:43.535307 17165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:43.546356 17177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:43.546402 17175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:43.546430 17174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:43.547464 17165 server_base.cc:1034] running on GCE node
I20260812 07:02:43.548282 17165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:43.549593 17165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:43.550814 17165 hybrid_clock.cc:648] HybridClock initialized: now 1786518163550760 us; error 33 us; skew 500 ppm
I20260812 07:02:43.551153 17165 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:43 dist-test-slave-lgr5 krb5kdc[17135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518163, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:43.556768 17165 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:02:43.558145 17165 webserver.cc:460] Webserver started at http://127.11.192.254:34843/ using document root <none> and password file <none>
I20260812 07:02:43.558809 17165 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:43.558940 17165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:43.559252 17165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:43.562288 17165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "9d607bdbee7440f9bfc8b5ee05840bf3"
format_stamp: "Formatted at 2026-08-12 07:02:43 on dist-test-slave-lgr5"
server_key: "7bb2b3e97652f280c6c8dd085f573839"
server_key_iv: "319b8e20a893cb39402010af63e40f8a"
server_key_version: "encryptionkey@0"
I20260812 07:02:43.563114 17165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "9d607bdbee7440f9bfc8b5ee05840bf3"
format_stamp: "Formatted at 2026-08-12 07:02:43 on dist-test-slave-lgr5"
server_key: "7bb2b3e97652f280c6c8dd085f573839"
server_key_iv: "319b8e20a893cb39402010af63e40f8a"
server_key_version: "encryptionkey@0"
I20260812 07:02:43.568236 17165 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260812 07:02:43.571411 17192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:43.572676 17165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:02:43.572825 17165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "9d607bdbee7440f9bfc8b5ee05840bf3"
format_stamp: "Formatted at 2026-08-12 07:02:43 on dist-test-slave-lgr5"
server_key: "7bb2b3e97652f280c6c8dd085f573839"
server_key_iv: "319b8e20a893cb39402010af63e40f8a"
server_key_version: "encryptionkey@0"
I20260812 07:02:43.573002 17165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:43.585178 17165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:43.588774 17165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:43.589025 17165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:43.605702 17165 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:39107
I20260812 07:02:43.605741 17283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:39107 every 8 connection(s)
I20260812 07:02:43.606911 17165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:02:43.610275 17285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:43.610929 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 17165
I20260812 07:02:43.611174 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:02:43.612023 12035 external_mini_cluster.cc:1428] Setting key 519899c35c78d8aaece2f722757d1213
I20260812 07:02:43.623564 17285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3: Bootstrap starting.
Aug 12 07:02:43 dist-test-slave-lgr5 krb5kdc[17135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:43.626823 17285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:43.627813 17285 log.cc:824] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:43.630283 17285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3: No bootstrap required, opened a new log
I20260812 07:02:43.633472 17297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:43.613819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38832 (local address 127.11.192.254:39107)
0812 07:02:43.614544 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:43.614554 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:43.614593 (+ 39us) server_negotiation.cc:407] Connection header received
0812 07:02:43.616386 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:43.616409 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:43.616766 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:43.617084 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:43.618845 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:43.620116 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:43.621793 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:43.622411 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:43.622528 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:43.626777 (+ 4249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:43.626784 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:43.626796 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:43.626836 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:43.629663 (+ 2827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:43.630280 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:43.630287 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:43.630293 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:43.630487 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:43.630841 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:43.630847 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:43.630850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:43.631393 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:43.631663 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:43.632145 (+ 482us) server_negotiation.cc:299] Negotiation successful
0812 07:02:43.632498 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":328,"threads_started":1}
I20260812 07:02:43.636904 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:02:43.637805 17285 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d607bdbee7440f9bfc8b5ee05840bf3" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39107 } }
I20260812 07:02:43.638119 17285 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:43.638211 17285 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d607bdbee7440f9bfc8b5ee05840bf3, State: Initialized, Role: FOLLOWER
I20260812 07:02:43.638689 17285 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d607bdbee7440f9bfc8b5ee05840bf3" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39107 } }
I20260812 07:02:43.638890 17285 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:02:43.639015 17285 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:02:43.639154 17285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:43.641599 17285 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d607bdbee7440f9bfc8b5ee05840bf3" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39107 } }
I20260812 07:02:43.642150 17285 leader_election.cc:302] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d607bdbee7440f9bfc8b5ee05840bf3; no voters:
I20260812 07:02:43.644347 17285 leader_election.cc:288] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:02:43.644434 17304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:02:43.645569 17304 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3 [term 1 LEADER]: Becoming Leader. State: Replica: 9d607bdbee7440f9bfc8b5ee05840bf3, State: Running, Role: LEADER
I20260812 07:02:43.646194 17304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d607bdbee7440f9bfc8b5ee05840bf3" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39107 } }
I20260812 07:02:43.647642 17285 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:02:43.649506 17305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d607bdbee7440f9bfc8b5ee05840bf3. Latest consensus state: current_term: 1 leader_uuid: "9d607bdbee7440f9bfc8b5ee05840bf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d607bdbee7440f9bfc8b5ee05840bf3" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39107 } } }
I20260812 07:02:43.649529 17306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d607bdbee7440f9bfc8b5ee05840bf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d607bdbee7440f9bfc8b5ee05840bf3" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39107 } } }
I20260812 07:02:43.649865 17305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3 [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:43.649870 17306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3 [sys.catalog]: This master's current role is: LEADER
W20260812 07:02:43.663939 17326 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:02:43.664105 17326 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:02:43.664232 17311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:02:43.665791 17311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:02:43.674909 17311 catalog_manager.cc:1347] Generated new cluster ID: 3e1b1d76c56b463d915e6e21dda4ab2d
I20260812 07:02:43.675020 17311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:02:43.687445 17311 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:02:43.688908 17311 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:02:43.694468 17297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:43.685727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38834 (local address 127.11.192.254:39107)
0812 07:02:43.685877 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:43.685881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:43.686449 (+ 568us) server_negotiation.cc:407] Connection header received
0812 07:02:43.686658 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:43.686663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:43.686742 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:43.686835 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:43.688157 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:43.689192 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:43.691015 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:43.691661 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:43.691779 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:43.692967 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:43.692973 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:43.692978 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:02:43.693695 (+ 717us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:43.693763 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:02:43.693793 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:43.693884 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:43.694034 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 07:02:43.694150 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 07:02:43.701974 17311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d607bdbee7440f9bfc8b5ee05840bf3: Generated new TSK 0
I20260812 07:02:43.702704 17311 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:02:43.713716 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 17165
I20260812 07:02:43.715380 17277 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:43.802047 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 17165
2026-08-12T07:02:43Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2001 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:02:43 dist-test-slave-lgr5 krb5kdc[17348](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:02:43 dist-test-slave-lgr5 krb5kdc[17348](info): set up 2 sockets
Aug 12 07:02:43 dist-test-slave-lgr5 krb5kdc[17348](info): commencing operation
krb5kdc: starting...
W20260812 07:02:45.061663 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260812 07:02:45.240470 12035 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.179s user 0.000s sys 0.007s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260812 07:02:45.423645 12035 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.183s user 0.001s sys 0.007s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260812 07:02:45.621629 12035 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.198s user 0.004s sys 0.004s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/test-user.keytab.
W20260812 07:02:46.065166 12035 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.443s user 0.000s sys 0.007s
Aug 12 07:02:46 dist-test-slave-lgr5 krb5kdc[17348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:02:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:02:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
W20260812 07:02:46.746392 12035 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.11.192.254: real 0.625s user 0.000s sys 0.012s
I20260812 07:02:46.747411 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:43953
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:32795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:43953
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:47.057942 17371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:47.058211 17371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:47.058346 17371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:47.065202 17371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:02:47.065308 17371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:47.065392 17371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:47.065465 17371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:02:47.065523 17371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:02:47.074767 17371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:32795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:43953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:43953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:47.075965 17371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:47.077188 17371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:47.087970 17380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:47.087976 17379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:47.088203 17384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:47.088778 17371 server_base.cc:1034] running on GCE node
I20260812 07:02:47.089452 17371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:47.090677 17371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:47.091866 17371 hybrid_clock.cc:648] HybridClock initialized: now 1786518167091815 us; error 42 us; skew 500 ppm
I20260812 07:02:47.092207 17371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:47 dist-test-slave-lgr5 krb5kdc[17348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518167, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:47.095170 17371 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:02:47.096510 17371 webserver.cc:460] Webserver started at http://127.11.192.254:43313/ using document root <none> and password file <none>
I20260812 07:02:47.097096 17371 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:47.097193 17371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:47.097441 17371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:47.100283 17371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "6a6589049c7f41f6baf05b394dbf8eba"
format_stamp: "Formatted at 2026-08-12 07:02:47 on dist-test-slave-lgr5"
server_key: "6152697abf8247d59e147903ba9af52a"
server_key_iv: "5e3c5af0d1fb7c4ce50ab7e5a773fb98"
server_key_version: "encryptionkey@0"
I20260812 07:02:47.100979 17371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "6a6589049c7f41f6baf05b394dbf8eba"
format_stamp: "Formatted at 2026-08-12 07:02:47 on dist-test-slave-lgr5"
server_key: "6152697abf8247d59e147903ba9af52a"
server_key_iv: "5e3c5af0d1fb7c4ce50ab7e5a773fb98"
server_key_version: "encryptionkey@0"
I20260812 07:02:47.105671 17371 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:02:47.108822 17398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:47.110021 17371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:02:47.110158 17371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "6a6589049c7f41f6baf05b394dbf8eba"
format_stamp: "Formatted at 2026-08-12 07:02:47 on dist-test-slave-lgr5"
server_key: "6152697abf8247d59e147903ba9af52a"
server_key_iv: "5e3c5af0d1fb7c4ce50ab7e5a773fb98"
server_key_version: "encryptionkey@0"
I20260812 07:02:47.110304 17371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:47.125136 17371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:47.129031 17371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:47.129244 17371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:47.145835 17371 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:43953
I20260812 07:02:47.145865 17489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:43953 every 8 connection(s)
I20260812 07:02:47.147038 17371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:02:47.150444 17490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:47.155373 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 17371
I20260812 07:02:47.155580 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:02:47.156240 12035 external_mini_cluster.cc:1428] Setting key 4b78435095a86dffb43e532990b0df00
I20260812 07:02:47.160858 17490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba: Bootstrap starting.
I20260812 07:02:47.164426 17490 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:47.165657 17490 log.cc:824] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:47.169122 17490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba: No bootstrap required, opened a new log
Aug 12 07:02:47 dist-test-slave-lgr5 krb5kdc[17348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:47.177636 17490 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a6589049c7f41f6baf05b394dbf8eba" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43953 } }
I20260812 07:02:47.178001 17490 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:47.178102 17490 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a6589049c7f41f6baf05b394dbf8eba, State: Initialized, Role: FOLLOWER
I20260812 07:02:47.178622 17490 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a6589049c7f41f6baf05b394dbf8eba" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43953 } }
I20260812 07:02:47.178848 17493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:47.158072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44320 (local address 127.11.192.254:43953)
0812 07:02:47.158901 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:47.158912 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:47.158951 (+ 39us) server_negotiation.cc:407] Connection header received
0812 07:02:47.160616 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:47.160638 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:47.160987 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:47.161306 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:47.163003 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:47.164354 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:47.166044 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:47.167061 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:47.167179 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:47.171866 (+ 4687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:47.171874 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:47.171886 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:47.171924 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:47.174843 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:47.175646 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:47.175652 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:47.175658 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:47.175825 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:47.176237 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:47.176240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:47.176242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:47.176761 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:47.176973 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:47.177388 (+ 415us) server_negotiation.cc:299] Negotiation successful
0812 07:02:47.177721 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":376,"threads_started":1}
I20260812 07:02:47.178853 17490 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:02:47.179059 17490 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:02:47.179248 17490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:47.182075 17490 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a6589049c7f41f6baf05b394dbf8eba" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43953 } }
I20260812 07:02:47.182512 17490 leader_election.cc:302] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a6589049c7f41f6baf05b394dbf8eba; no voters:
I20260812 07:02:47.183007 17490 leader_election.cc:288] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:02:47.183148 17496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:02:47.184154 17496 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba [term 1 LEADER]: Becoming Leader. State: Replica: 6a6589049c7f41f6baf05b394dbf8eba, State: Running, Role: LEADER
I20260812 07:02:47.184630 17496 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a6589049c7f41f6baf05b394dbf8eba" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43953 } }
I20260812 07:02:47.184937 17490 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:02:47.187062 17497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a6589049c7f41f6baf05b394dbf8eba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a6589049c7f41f6baf05b394dbf8eba" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43953 } } }
I20260812 07:02:47.187394 17497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:47.187012 17499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a6589049c7f41f6baf05b394dbf8eba. Latest consensus state: current_term: 1 leader_uuid: "6a6589049c7f41f6baf05b394dbf8eba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a6589049c7f41f6baf05b394dbf8eba" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43953 } } }
I20260812 07:02:47.187662 17499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:47.188369 17507 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:02:47.193598 17507 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:02:47.202070 17507 catalog_manager.cc:1347] Generated new cluster ID: 2c873eff113443e99e5543d3ccb6618a
I20260812 07:02:47.202170 17507 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:02:47.239368 17507 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:02:47.241250 17507 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:02:47.249962 17507 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a6589049c7f41f6baf05b394dbf8eba: Generated new TSK 0
I20260812 07:02:47.250717 17507 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
I20260812 07:02:47.305922 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43953
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:32795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:47.617772 17532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:47.618045 17532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:47.618178 17532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:47.625095 17532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:47.625200 17532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:47.625341 17532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:02:47.634433 17532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:32795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:47.635564 17532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:47.636794 17532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:47.649070 17542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:47.649003 17543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:47.649132 17550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:47.650012 17532 server_base.cc:1034] running on GCE node
I20260812 07:02:47.650553 17532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:47.651386 17532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:47.652583 17532 hybrid_clock.cc:648] HybridClock initialized: now 1786518167652543 us; error 29 us; skew 500 ppm
I20260812 07:02:47.652871 17532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:47 dist-test-slave-lgr5 krb5kdc[17348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518167, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:47.655725 17532 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:02:47.657105 17532 webserver.cc:460] Webserver started at http://127.11.192.193:41567/ using document root <none> and password file <none>
I20260812 07:02:47.657734 17532 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:47.657852 17532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:47.658135 17532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:47.660853 17532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "d9b3f014dfc644b99ae20ba61bec3d5a"
format_stamp: "Formatted at 2026-08-12 07:02:47 on dist-test-slave-lgr5"
server_key: "c5835615d9c4a5e73a786fb047fc776b"
server_key_iv: "476a180b9c94b049e665a9b37d7fc426"
server_key_version: "encryptionkey@0"
I20260812 07:02:47.661612 17532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "d9b3f014dfc644b99ae20ba61bec3d5a"
format_stamp: "Formatted at 2026-08-12 07:02:47 on dist-test-slave-lgr5"
server_key: "c5835615d9c4a5e73a786fb047fc776b"
server_key_iv: "476a180b9c94b049e665a9b37d7fc426"
server_key_version: "encryptionkey@0"
I20260812 07:02:47.666421 17532 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 07:02:47.669589 17564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:47.670764 17532 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:02:47.670917 17532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "d9b3f014dfc644b99ae20ba61bec3d5a"
format_stamp: "Formatted at 2026-08-12 07:02:47 on dist-test-slave-lgr5"
server_key: "c5835615d9c4a5e73a786fb047fc776b"
server_key_iv: "476a180b9c94b049e665a9b37d7fc426"
server_key_version: "encryptionkey@0"
I20260812 07:02:47.671092 17532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:47.690725 17532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:47.694329 17532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:47.694586 17532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:47.695498 17532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:02:47.696765 17532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:47.696842 17532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:47.696962 17532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:47.697022 17532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:47.726642 17532 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:40799
I20260812 07:02:47.726686 17758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:40799 every 8 connection(s)
I20260812 07:02:47.727818 17532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:02:47.733711 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 17532
I20260812 07:02:47.733953 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:02:47.734763 12035 external_mini_cluster.cc:1428] Setting key efa97c3ff3ee8fcd1052459a6dd65d41
Aug 12 07:02:47 dist-test-slave-lgr5 krb5kdc[17348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518167, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:47.749316 17761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:47.730180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:50539 (local address 127.11.192.254:43953)
0812 07:02:47.730616 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:47.730623 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:47.731535 (+ 912us) server_negotiation.cc:407] Connection header received
0812 07:02:47.733302 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:47.733306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:47.733371 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:47.733454 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:47.735446 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:47.736460 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:47.738505 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:47.738897 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:47.739000 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:47.743101 (+ 4101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:47.743107 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:47.743113 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:47.743155 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:47.746105 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:47.746959 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:47.746963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:47.746965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:47.747092 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:47.747562 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:47.747565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:47.747567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:47.747831 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:47.747941 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:47.748803 (+ 862us) server_negotiation.cc:299] Negotiation successful
0812 07:02:47.749001 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":207,"threads_started":1}
I20260812 07:02:47.750681 17762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:47.730723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43953 (local address 127.11.192.193:50539)
0812 07:02:47.731384 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:47.731420 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:47.733055 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:47.733751 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:47.733762 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:47.734468 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:47.735263 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:47.735291 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:47.736606 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:47.736611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:47.738362 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:47.738379 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:47.739107 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:47.739112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:47.739371 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:47.740052 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:47.740107 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:47.742863 (+ 2756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:47.746265 (+ 3402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:47.746275 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:47.746289 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:47.746837 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:47.747198 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:47.747204 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:47.747208 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:47.747460 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:47.747958 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:47.747967 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:47.748392 (+ 425us) client_negotiation.cc:769] Sending connection context
0812 07:02:47.748757 (+ 365us) client_negotiation.cc:240] Negotiation successful
0812 07:02:47.749273 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":249,"threads_started":1}
I20260812 07:02:47.752247 17759 heartbeater.cc:347] Connected to a master server at 127.11.192.254:43953
I20260812 07:02:47.752521 17759 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:47.753203 17759 heartbeater.cc:511] Master 127.11.192.254:43953 requested a full tablet report, sending...
I20260812 07:02:47.755102 17428 ts_manager.cc:194] Registered new tserver with Master: d9b3f014dfc644b99ae20ba61bec3d5a (127.11.192.193:40799)
I20260812 07:02:47.756537 17428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:50539
WARNING: no policy specified for kudu/127.11.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
I20260812 07:02:47.780999 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43953
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:32795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:48.090931 17766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:48.091169 17766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:48.091264 17766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:48.098210 17766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:48.098301 17766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:48.098412 17766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:02:48.107713 17766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:32795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:48.109030 17766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:48.110251 17766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:48.122370 17782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:48.122442 17787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:48.122601 17790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:48.123207 17766 server_base.cc:1034] running on GCE node
I20260812 07:02:48.123739 17766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:48.124639 17766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:48.125834 17766 hybrid_clock.cc:648] HybridClock initialized: now 1786518168125795 us; error 27 us; skew 500 ppm
I20260812 07:02:48.126122 17766 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:48 dist-test-slave-lgr5 krb5kdc[17348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518168, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:48.129040 17766 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:02:48.130306 17766 webserver.cc:460] Webserver started at http://127.11.192.194:38605/ using document root <none> and password file <none>
I20260812 07:02:48.130916 17766 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:48.131042 17766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:48.131325 17766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:48.134081 17766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "901fa9f9955b4d3db67f5aef060c864d"
format_stamp: "Formatted at 2026-08-12 07:02:48 on dist-test-slave-lgr5"
server_key: "6f59f5862b4fc18236abac2f121c5845"
server_key_iv: "3c5599e7938bb8ff783886c3fc355f37"
server_key_version: "encryptionkey@0"
I20260812 07:02:48.134815 17766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "901fa9f9955b4d3db67f5aef060c864d"
format_stamp: "Formatted at 2026-08-12 07:02:48 on dist-test-slave-lgr5"
server_key: "6f59f5862b4fc18236abac2f121c5845"
server_key_iv: "3c5599e7938bb8ff783886c3fc355f37"
server_key_version: "encryptionkey@0"
I20260812 07:02:48.139528 17766 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260812 07:02:48.142838 17799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:48.143960 17766 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:02:48.144135 17766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "901fa9f9955b4d3db67f5aef060c864d"
format_stamp: "Formatted at 2026-08-12 07:02:48 on dist-test-slave-lgr5"
server_key: "6f59f5862b4fc18236abac2f121c5845"
server_key_iv: "3c5599e7938bb8ff783886c3fc355f37"
server_key_version: "encryptionkey@0"
I20260812 07:02:48.144299 17766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:48.170495 17766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:48.174069 17766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:48.174336 17766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:48.175267 17766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:02:48.176636 17766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:48.176715 17766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:48.176831 17766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:48.176883 17766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:48.206264 17766 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:36729
I20260812 07:02:48.206322 18023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:36729 every 8 connection(s)
I20260812 07:02:48.207496 17766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:02:48.217795 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 17766
I20260812 07:02:48.217970 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:02:48.218570 12035 external_mini_cluster.cc:1428] Setting key 4573dfac0165eba81c8186053836726f
Aug 12 07:02:48 dist-test-slave-lgr5 krb5kdc[17348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518168, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:48.226094 17761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:48.209806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:44911 (local address 127.11.192.254:43953)
0812 07:02:48.209970 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:48.209974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:48.211028 (+ 1054us) server_negotiation.cc:407] Connection header received
0812 07:02:48.212828 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:48.212833 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:48.212910 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:48.213011 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:48.214425 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:48.215217 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:48.216673 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:48.217069 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:48.217159 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:48.220846 (+ 3687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:48.220850 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:48.220854 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:48.220882 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:48.223275 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:48.223931 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:48.223935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:48.223937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:48.224068 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:48.224515 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:48.224519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:48.224521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:48.224756 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:48.224889 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:48.225550 (+ 661us) server_negotiation.cc:299] Negotiation successful
0812 07:02:48.225777 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:02:48.226886 18028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:48.210213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43953 (local address 127.11.192.194:44911)
0812 07:02:48.210879 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:48.210922 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:48.212571 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:48.213237 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:48.213245 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:48.213736 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:48.214279 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:48.214297 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:48.215322 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:48.215326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:48.216567 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:48.216579 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:48.217228 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:48.217233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:48.217419 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:48.218112 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:48.218141 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:48.220655 (+ 2514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:48.223402 (+ 2747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:48.223411 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:48.223426 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:48.223810 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:48.224209 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:48.224214 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:48.224219 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:48.224392 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:48.224890 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:48.224897 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:48.225183 (+ 286us) client_negotiation.cc:769] Sending connection context
0812 07:02:48.225394 (+ 211us) client_negotiation.cc:240] Negotiation successful
0812 07:02:48.225849 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":256,"threads_started":1}
I20260812 07:02:48.228192 18025 heartbeater.cc:347] Connected to a master server at 127.11.192.254:43953
I20260812 07:02:48.228449 18025 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:48.229095 18025 heartbeater.cc:511] Master 127.11.192.254:43953 requested a full tablet report, sending...
I20260812 07:02:48.230438 17428 ts_manager.cc:194] Registered new tserver with Master: 901fa9f9955b4d3db67f5aef060c864d (127.11.192.194:36729)
I20260812 07:02:48.231194 17428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:44911
WARNING: no policy specified for kudu/127.11.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
I20260812 07:02:48.260960 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43953
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:32795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:48.580214 18036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:48.580461 18036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:48.580577 18036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:48.587596 18036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:48.587687 18036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:48.587824 18036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:02:48.597106 18036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:32795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:48.598227 18036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:48.599407 18036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:48.613806 18047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:48.613857 18048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:48.616298 18051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:48.621433 18036 server_base.cc:1034] running on GCE node
I20260812 07:02:48.621969 18036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:48.622859 18036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:48.624069 18036 hybrid_clock.cc:648] HybridClock initialized: now 1786518168624025 us; error 51 us; skew 500 ppm
I20260812 07:02:48.624388 18036 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:48 dist-test-slave-lgr5 krb5kdc[17348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518168, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:48.630748 18036 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:02:48.632241 18036 webserver.cc:460] Webserver started at http://127.11.192.195:43767/ using document root <none> and password file <none>
I20260812 07:02:48.632886 18036 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:48.632992 18036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:48.633275 18036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:48.636199 18036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "b77fe64abf254eb2a178eabfcdce2a75"
format_stamp: "Formatted at 2026-08-12 07:02:48 on dist-test-slave-lgr5"
server_key: "a4c285a5d574358620c8da9bc875dee9"
server_key_iv: "63727a05cc45e28a883d55cc8fcb6626"
server_key_version: "encryptionkey@0"
I20260812 07:02:48.636936 18036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "b77fe64abf254eb2a178eabfcdce2a75"
format_stamp: "Formatted at 2026-08-12 07:02:48 on dist-test-slave-lgr5"
server_key: "a4c285a5d574358620c8da9bc875dee9"
server_key_iv: "63727a05cc45e28a883d55cc8fcb6626"
server_key_version: "encryptionkey@0"
I20260812 07:02:48.641754 18036 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:02:48.644963 18067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:48.646303 18036 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 07:02:48.646443 18036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "b77fe64abf254eb2a178eabfcdce2a75"
format_stamp: "Formatted at 2026-08-12 07:02:48 on dist-test-slave-lgr5"
server_key: "a4c285a5d574358620c8da9bc875dee9"
server_key_iv: "63727a05cc45e28a883d55cc8fcb6626"
server_key_version: "encryptionkey@0"
I20260812 07:02:48.646620 18036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:48.683977 18036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:48.688269 18036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:48.688541 18036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:48.689495 18036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:02:48.690876 18036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:48.690959 18036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:48.691076 18036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:48.691124 18036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:48.734506 18036 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:38129
I20260812 07:02:48.734541 18280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:38129 every 8 connection(s)
I20260812 07:02:48.735685 18036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:02:48.737462 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 18036
I20260812 07:02:48.737660 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:02:48.738162 12035 external_mini_cluster.cc:1428] Setting key 8ee8af8fff5e1fac0ae2f0b1e25ff4c3
Aug 12 07:02:48 dist-test-slave-lgr5 krb5kdc[17348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518168, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:48.754184 18286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:48.738259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:32933 (local address 127.11.192.254:43953)
0812 07:02:48.738737 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:48.738742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:48.739805 (+ 1063us) server_negotiation.cc:407] Connection header received
0812 07:02:48.741619 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:48.741623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:48.741685 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:48.741762 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:48.743069 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:48.743823 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:48.745280 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:48.745662 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:48.745751 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:48.748842 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:48.748847 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:48.748850 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:48.748894 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:48.751272 (+ 2378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:48.751976 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:48.751981 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:48.751993 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:48.752145 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:48.752606 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:48.752610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:48.752612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:48.752964 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:48.753114 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:48.753773 (+ 659us) server_negotiation.cc:299] Negotiation successful
0812 07:02:48.753926 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":241,"threads_started":1}
I20260812 07:02:48.755077 18287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:48.738825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43953 (local address 127.11.192.195:32933)
0812 07:02:48.739647 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:48.739692 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:48.741371 (+ 1679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:48.741921 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:48.741928 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:48.742394 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:48.742917 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:48.742940 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:48.743931 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:48.743934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:48.745174 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:48.745187 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:48.745821 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:48.745826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:48.746013 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:48.746560 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:48.746581 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:48.748648 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:48.751403 (+ 2755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:48.751410 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:48.751419 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:48.751867 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:48.752260 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:48.752265 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:48.752269 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:48.752496 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:48.753114 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:48.753122 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:48.753458 (+ 336us) client_negotiation.cc:769] Sending connection context
0812 07:02:48.753689 (+ 231us) client_negotiation.cc:240] Negotiation successful
0812 07:02:48.754047 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":304,"threads_started":1}
I20260812 07:02:48.756337 18283 heartbeater.cc:347] Connected to a master server at 127.11.192.254:43953
I20260812 07:02:48.756608 18283 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:48.757283 18283 heartbeater.cc:511] Master 127.11.192.254:43953 requested a full tablet report, sending...
I20260812 07:02:48.758634 17428 ts_manager.cc:194] Registered new tserver with Master: b77fe64abf254eb2a178eabfcdce2a75 (127.11.192.195:38129)
I20260812 07:02:48.759123 17759 heartbeater.cc:503] Master 127.11.192.254:43953 was elected leader, sending a full tablet report...
I20260812 07:02:48.759460 17428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:32933
I20260812 07:02:48.765084 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 07:02:48 dist-test-slave-lgr5 krb5kdc[17348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518168, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 07:02:48 dist-test-slave-lgr5 krb5kdc[17348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518168, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:48.815508 18286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:48.802005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44334 (local address 127.11.192.254:43953)
0812 07:02:48.802246 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:48.802254 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:48.802642 (+ 388us) server_negotiation.cc:407] Connection header received
0812 07:02:48.802808 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:48.802811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:48.802893 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:48.802979 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:48.804406 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:48.805126 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:48.806412 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:48.807018 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:48.807135 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:48.810822 (+ 3687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:48.810827 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:48.810830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:48.810860 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:48.813336 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:48.813912 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:48.813917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:48.813921 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:48.814062 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:48.814401 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:48.814404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:48.814406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:48.814711 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:48.814889 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:48.815123 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 07:02:48.815281 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260812 07:02:48.816046 17427 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44334
I20260812 07:02:48.817751 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 17532
Aug 12 07:02:48 dist-test-slave-lgr5 krb5kdc[17348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518168, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.192.193@KRBTEST.COM
I20260812 07:02:48.833362 18303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:48.818895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37208 (local address 127.11.192.193:40799)
0812 07:02:48.819479 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:48.819486 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:48.819511 (+ 25us) server_negotiation.cc:407] Connection header received
0812 07:02:48.819564 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:48.819571 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:48.819770 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:48.819901 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:48.821041 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:48.822031 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:48.823162 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:48.823693 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:48.823783 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:48.827452 (+ 3669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:48.827464 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:48.827478 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:48.827513 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:48.830460 (+ 2947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:48.831196 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:48.831205 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:48.831209 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:48.831369 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:48.831800 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:48.831805 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:48.831809 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:48.832208 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:48.832504 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:48.832774 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 07:02:48.833032 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":285,"threads_started":1}
W20260812 07:02:48.834589 17750 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37208
W20260812 07:02:48.835603 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 17532: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:02:48.835758 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 17532
I20260812 07:02:48.861536 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 17766
Aug 12 07:02:48 dist-test-slave-lgr5 krb5kdc[17348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518168, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.192.194@KRBTEST.COM
I20260812 07:02:48.875787 18304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:48.862338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38746 (local address 127.11.192.194:36729)
0812 07:02:48.862808 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:48.862816 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:48.862839 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:02:48.862899 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:48.862905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:48.863119 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:48.863243 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:48.864428 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:48.865392 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:48.866507 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:48.867023 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:48.867110 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:48.870954 (+ 3844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:48.870964 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:48.870977 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:48.871018 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:48.873537 (+ 2519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:48.874096 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:48.874102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:48.874104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:48.874222 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:48.874477 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:48.874480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:48.874482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:48.874795 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:48.875027 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:48.875246 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 07:02:48.875438 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":244,"threads_started":1}
W20260812 07:02:48.877059 18014 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38746
W20260812 07:02:48.878039 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 17766: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:02:48.878188 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 17766
I20260812 07:02:48.903925 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 18036
Aug 12 07:02:48 dist-test-slave-lgr5 krb5kdc[17348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518168, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.192.195@KRBTEST.COM
I20260812 07:02:48.919001 18305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:48.904903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50426 (local address 127.11.192.195:38129)
0812 07:02:48.905526 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:48.905533 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:48.905560 (+ 27us) server_negotiation.cc:407] Connection header received
0812 07:02:48.905621 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:48.905627 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:48.905824 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:48.905942 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:48.907195 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:48.908337 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:48.909445 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:48.909998 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:48.910089 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:48.914019 (+ 3930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:48.914030 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:48.914045 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:48.914082 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:48.916833 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:48.917387 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:48.917393 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:48.917395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:48.917512 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:48.917771 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:48.917774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:48.917776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:48.918097 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:48.918323 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:48.918520 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 07:02:48.918708 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":309,"threads_started":1}
W20260812 07:02:48.920302 18274 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50426
W20260812 07:02:48.921315 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 18036: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:02:48.921442 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 18036
I20260812 07:02:48.943246 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 17371
I20260812 07:02:48.944268 17481 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:49.044059 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 17371
2026-08-12T07:02:49Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (5249 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:02:49 dist-test-slave-lgr5 krb5kdc[18312](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:02:49 dist-test-slave-lgr5 krb5kdc[18312](info): set up 2 sockets
Aug 12 07:02:49 dist-test-slave-lgr5 krb5kdc[18312](info): commencing operation
krb5kdc: starting...
W20260812 07:02:50.263258 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.007s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:02:50 dist-test-slave-lgr5 krb5kdc[18312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:02:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:02:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:02:50.448841 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:35835
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:35835
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:50.755806 18330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:50.756054 18330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:50.756191 18330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:50.762990 18330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:02:50.763075 18330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:50.763144 18330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:50.763213 18330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:02:50.763268 18330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:02:50.772379 18330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:35835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:35835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:50.773500 18330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:50.774663 18330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:50.785233 18340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:50.785270 18345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:50.785311 18343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:50.786288 18330 server_base.cc:1034] running on GCE node
I20260812 07:02:50.786955 18330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:50.788216 18330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:50.789412 18330 hybrid_clock.cc:648] HybridClock initialized: now 1786518170789366 us; error 36 us; skew 500 ppm
I20260812 07:02:50.789711 18330 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:50 dist-test-slave-lgr5 krb5kdc[18312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518170, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:50.792572 18330 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:02:50.793846 18330 webserver.cc:460] Webserver started at http://127.11.192.254:37845/ using document root <none> and password file <none>
I20260812 07:02:50.794447 18330 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:50.794566 18330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:50.794845 18330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:50.797614 18330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "39bdd56373d44be5bb2e8a31d06586b1"
format_stamp: "Formatted at 2026-08-12 07:02:50 on dist-test-slave-lgr5"
server_key: "2820770c746794b468762f1a26085b09"
server_key_iv: "f3cd55900bf498547a7b8c1e13844740"
server_key_version: "encryptionkey@0"
I20260812 07:02:50.798360 18330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "39bdd56373d44be5bb2e8a31d06586b1"
format_stamp: "Formatted at 2026-08-12 07:02:50 on dist-test-slave-lgr5"
server_key: "2820770c746794b468762f1a26085b09"
server_key_iv: "f3cd55900bf498547a7b8c1e13844740"
server_key_version: "encryptionkey@0"
I20260812 07:02:50.803054 18330 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 07:02:50.806280 18360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:50.807422 18330 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 07:02:50.807583 18330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "39bdd56373d44be5bb2e8a31d06586b1"
format_stamp: "Formatted at 2026-08-12 07:02:50 on dist-test-slave-lgr5"
server_key: "2820770c746794b468762f1a26085b09"
server_key_iv: "f3cd55900bf498547a7b8c1e13844740"
server_key_version: "encryptionkey@0"
I20260812 07:02:50.807756 18330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:50.831190 18330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:50.834736 18330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:50.834971 18330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:50.852442 18330 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:35835
I20260812 07:02:50.852466 18450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:35835 every 8 connection(s)
I20260812 07:02:50.853720 18330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:02:50.857199 18451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:50.862622 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 18330
I20260812 07:02:50.862790 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:02:50.863396 12035 external_mini_cluster.cc:1428] Setting key 020a5d265e4dbe9e425c05300c227123
I20260812 07:02:50.868052 18451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1: Bootstrap starting.
I20260812 07:02:50.871623 18451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:50.872852 18451 log.cc:824] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:50.875842 18451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1: No bootstrap required, opened a new log
Aug 12 07:02:50 dist-test-slave-lgr5 krb5kdc[18312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:50.884838 18451 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bdd56373d44be5bb2e8a31d06586b1" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35835 } }
I20260812 07:02:50.885272 18451 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:50.885452 18451 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39bdd56373d44be5bb2e8a31d06586b1, State: Initialized, Role: FOLLOWER
I20260812 07:02:50.886063 18451 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bdd56373d44be5bb2e8a31d06586b1" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35835 } }
I20260812 07:02:50.886286 18451 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:02:50.886423 18451 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:02:50.886543 18456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:50.865332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60532 (local address 127.11.192.254:35835)
0812 07:02:50.866166 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:50.866180 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:50.866226 (+ 46us) server_negotiation.cc:407] Connection header received
0812 07:02:50.868504 (+ 2278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:50.868531 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:50.868981 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:50.869370 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:50.870618 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:50.872582 (+ 1964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:50.874113 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:50.875112 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:50.875282 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:50.878860 (+ 3578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:50.878871 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:50.878889 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:50.878937 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:50.882761 (+ 3824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:50.883384 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:50.883392 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:50.883400 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:50.883632 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:50.883934 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:50.883939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:50.883943 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:50.884637 (+ 694us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:50.884910 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:50.885176 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 07:02:50.885558 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":348,"threads_started":1}
I20260812 07:02:50.886600 18451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:50.889357 18451 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bdd56373d44be5bb2e8a31d06586b1" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35835 } }
I20260812 07:02:50.889846 18451 leader_election.cc:302] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39bdd56373d44be5bb2e8a31d06586b1; no voters:
I20260812 07:02:50.890363 18451 leader_election.cc:288] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:02:50.890483 18460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:02:50.891112 18460 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1 [term 1 LEADER]: Becoming Leader. State: Replica: 39bdd56373d44be5bb2e8a31d06586b1, State: Running, Role: LEADER
I20260812 07:02:50.891604 18460 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bdd56373d44be5bb2e8a31d06586b1" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35835 } }
I20260812 07:02:50.892359 18451 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:02:50.894418 18461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39bdd56373d44be5bb2e8a31d06586b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bdd56373d44be5bb2e8a31d06586b1" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35835 } } }
I20260812 07:02:50.894433 18462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39bdd56373d44be5bb2e8a31d06586b1. Latest consensus state: current_term: 1 leader_uuid: "39bdd56373d44be5bb2e8a31d06586b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bdd56373d44be5bb2e8a31d06586b1" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35835 } } }
I20260812 07:02:50.894906 18462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1 [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:50.894936 18461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1 [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:50.897351 18466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:02:50.902160 18466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:02:50.910604 18466 catalog_manager.cc:1347] Generated new cluster ID: 14681cad45bc488cae6034a933af68fa
I20260812 07:02:50.910699 18466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:02:50.919118 18466 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:02:50.920789 18466 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:02:50.930893 18466 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39bdd56373d44be5bb2e8a31d06586b1: Generated new TSK 0
I20260812 07:02:50.931762 18466 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
I20260812 07:02:50.977357 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:35835
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:51.290105 18488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:51.290350 18488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:51.290484 18488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:51.297556 18488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:51.297653 18488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:51.297798 18488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:02:51.307003 18488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:35835
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:51.308099 18488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:51.309302 18488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:51.321682 18496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:51.321743 18498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:51.321897 18495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:51.322824 18488 server_base.cc:1034] running on GCE node
I20260812 07:02:51.323431 18488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:51.324456 18488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:51.325670 18488 hybrid_clock.cc:648] HybridClock initialized: now 1786518171325623 us; error 31 us; skew 500 ppm
I20260812 07:02:51.325968 18488 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:51 dist-test-slave-lgr5 krb5kdc[18312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518171, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:51.328891 18488 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:02:51.330209 18488 webserver.cc:460] Webserver started at http://127.11.192.193:36487/ using document root <none> and password file <none>
I20260812 07:02:51.330829 18488 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:51.330947 18488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:51.331233 18488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:51.333969 18488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "a00e21d0cbfb465085af28a1b8d9c30a"
format_stamp: "Formatted at 2026-08-12 07:02:51 on dist-test-slave-lgr5"
server_key: "07101e0049a794ecd96c7f6c46fe9588"
server_key_iv: "ddf603c1b2c17470384998bb5503ae96"
server_key_version: "encryptionkey@0"
I20260812 07:02:51.334687 18488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "a00e21d0cbfb465085af28a1b8d9c30a"
format_stamp: "Formatted at 2026-08-12 07:02:51 on dist-test-slave-lgr5"
server_key: "07101e0049a794ecd96c7f6c46fe9588"
server_key_iv: "ddf603c1b2c17470384998bb5503ae96"
server_key_version: "encryptionkey@0"
I20260812 07:02:51.339614 18488 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260812 07:02:51.342831 18510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:51.344100 18488 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:02:51.344254 18488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "a00e21d0cbfb465085af28a1b8d9c30a"
format_stamp: "Formatted at 2026-08-12 07:02:51 on dist-test-slave-lgr5"
server_key: "07101e0049a794ecd96c7f6c46fe9588"
server_key_iv: "ddf603c1b2c17470384998bb5503ae96"
server_key_version: "encryptionkey@0"
I20260812 07:02:51.344425 18488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:51.357821 18488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:51.361313 18488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:51.361554 18488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:51.362383 18488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:02:51.363603 18488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:51.363694 18488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:51.363801 18488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:51.363855 18488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:51.393404 18488 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:37377
I20260812 07:02:51.393463 18710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:37377 every 8 connection(s)
I20260812 07:02:51.394562 18488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:02:51.404578 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 18488
I20260812 07:02:51.404771 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:02:51.405488 12035 external_mini_cluster.cc:1428] Setting key 2d3a342a638dbec6f34655466cd4bfa2
Aug 12 07:02:51 dist-test-slave-lgr5 krb5kdc[18312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518171, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:51.414108 18715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:51.396967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:33911 (local address 127.11.192.254:35835)
0812 07:02:51.397382 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:51.397386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:51.398322 (+ 936us) server_negotiation.cc:407] Connection header received
0812 07:02:51.400150 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:51.400155 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:51.400232 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:51.400329 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:51.401752 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:51.402459 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:51.403956 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:51.404549 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:51.404850 (+ 301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:51.408215 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:51.408221 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:51.408225 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:51.408262 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:51.411174 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:51.411888 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:51.411893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:51.411897 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:51.412028 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:51.412466 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:51.412471 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:51.412473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:51.412747 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:51.412876 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:51.413523 (+ 647us) server_negotiation.cc:299] Negotiation successful
0812 07:02:51.413740 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":222,"threads_started":1}
I20260812 07:02:51.414939 18716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:51.397470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:35835 (local address 127.11.192.193:33911)
0812 07:02:51.398171 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:51.398218 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:51.399869 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:51.400569 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:51.400577 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:51.401057 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:51.401584 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:51.401603 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:51.402625 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:51.402629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:51.403813 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:51.403826 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:51.404916 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:51.404924 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:51.405137 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:51.405755 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:51.405776 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:51.407987 (+ 2211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:51.411308 (+ 3321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:51.411315 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:51.411324 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:51.411782 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:51.412148 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:51.412154 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:51.412158 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:51.412357 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:51.412885 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:51.412893 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:51.413184 (+ 291us) client_negotiation.cc:769] Sending connection context
0812 07:02:51.413421 (+ 237us) client_negotiation.cc:240] Negotiation successful
0812 07:02:51.413893 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":266,"threads_started":1}
I20260812 07:02:51.416345 18713 heartbeater.cc:347] Connected to a master server at 127.11.192.254:35835
I20260812 07:02:51.416616 18713 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:51.417245 18713 heartbeater.cc:511] Master 127.11.192.254:35835 requested a full tablet report, sending...
I20260812 07:02:51.419162 18391 ts_manager.cc:194] Registered new tserver with Master: a00e21d0cbfb465085af28a1b8d9c30a (127.11.192.193:37377)
I20260812 07:02:51.420584 18391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:33911
WARNING: no policy specified for kudu/127.11.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
I20260812 07:02:51.448776 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:35835
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:51.759299 18720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:51.759569 18720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:51.759675 18720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:51.766717 18720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:51.766809 18720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:51.766923 18720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:02:51.776132 18720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:35835
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:51.777190 18720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:51.778360 18720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:51.790531 18726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:51.790557 18731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:51.790481 18727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:51.791414 18720 server_base.cc:1034] running on GCE node
I20260812 07:02:51.791937 18720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:51.792852 18720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:51.794062 18720 hybrid_clock.cc:648] HybridClock initialized: now 1786518171794017 us; error 38 us; skew 500 ppm
I20260812 07:02:51.794342 18720 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:51 dist-test-slave-lgr5 krb5kdc[18312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518171, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:51.797248 18720 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:02:51.798553 18720 webserver.cc:460] Webserver started at http://127.11.192.194:41467/ using document root <none> and password file <none>
I20260812 07:02:51.799187 18720 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:51.799300 18720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:51.799587 18720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:51.802284 18720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "72198a7e45ac42cd901857b7cf6c7d82"
format_stamp: "Formatted at 2026-08-12 07:02:51 on dist-test-slave-lgr5"
server_key: "490afd024d5dac8c9d744321710eee14"
server_key_iv: "d3eb1a5b73589517ccf5f0a4ae21996f"
server_key_version: "encryptionkey@0"
I20260812 07:02:51.803027 18720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "72198a7e45ac42cd901857b7cf6c7d82"
format_stamp: "Formatted at 2026-08-12 07:02:51 on dist-test-slave-lgr5"
server_key: "490afd024d5dac8c9d744321710eee14"
server_key_iv: "d3eb1a5b73589517ccf5f0a4ae21996f"
server_key_version: "encryptionkey@0"
I20260812 07:02:51.807717 18720 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260812 07:02:51.810909 18742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:51.812176 18720 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:02:51.812325 18720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "72198a7e45ac42cd901857b7cf6c7d82"
format_stamp: "Formatted at 2026-08-12 07:02:51 on dist-test-slave-lgr5"
server_key: "490afd024d5dac8c9d744321710eee14"
server_key_iv: "d3eb1a5b73589517ccf5f0a4ae21996f"
server_key_version: "encryptionkey@0"
I20260812 07:02:51.812494 18720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:51.822804 18720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:51.826572 18720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:51.826790 18720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:51.827567 18720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:02:51.828792 18720 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:51.828862 18720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:51.828948 18720 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:51.828989 18720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:51.865967 18720 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:38429
I20260812 07:02:51.866007 18956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:38429 every 8 connection(s)
I20260812 07:02:51.867348 18720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:02:51.874913 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 18720
I20260812 07:02:51.875078 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:02:51.875607 12035 external_mini_cluster.cc:1428] Setting key 6320d728677786a6b75e690b5b24c43e
Aug 12 07:02:51 dist-test-slave-lgr5 krb5kdc[18312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518171, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:51.887422 18715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:51.869773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:56717 (local address 127.11.192.254:35835)
0812 07:02:51.869949 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:51.869954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:51.871098 (+ 1144us) server_negotiation.cc:407] Connection header received
0812 07:02:51.872840 (+ 1742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:51.872844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:51.872930 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:51.873026 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:51.874348 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:51.875140 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:51.877018 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:51.877655 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:51.877764 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:51.881741 (+ 3977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:51.881747 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:51.881750 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:51.881786 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:51.884306 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:51.885108 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:51.885112 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:51.885114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:51.885264 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:51.885683 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:51.885686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:51.885688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:51.885961 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:51.886091 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:51.886777 (+ 686us) server_negotiation.cc:299] Negotiation successful
0812 07:02:51.887014 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:02:51.887948 18962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:51.870235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:35835 (local address 127.11.192.194:56717)
0812 07:02:51.870926 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:51.870976 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:51.872586 (+ 1610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:51.873196 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:51.873203 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:51.873678 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:51.874213 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:51.874229 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:51.875313 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:51.875319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:51.876870 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:51.876888 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:51.877891 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:51.877899 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:51.878193 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:51.878871 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:51.878898 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:51.881507 (+ 2609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:51.884452 (+ 2945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:51.884460 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:51.884474 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:51.884987 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:51.885366 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:51.885371 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:51.885374 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:51.885589 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:51.886078 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:51.886085 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:51.886407 (+ 322us) client_negotiation.cc:769] Sending connection context
0812 07:02:51.886614 (+ 207us) client_negotiation.cc:240] Negotiation successful
0812 07:02:51.886951 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":270,"threads_started":1}
I20260812 07:02:51.889278 18958 heartbeater.cc:347] Connected to a master server at 127.11.192.254:35835
I20260812 07:02:51.889557 18958 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:51.890202 18958 heartbeater.cc:511] Master 127.11.192.254:35835 requested a full tablet report, sending...
I20260812 07:02:51.891566 18391 ts_manager.cc:194] Registered new tserver with Master: 72198a7e45ac42cd901857b7cf6c7d82 (127.11.192.194:38429)
I20260812 07:02:51.892441 18391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:56717
WARNING: no policy specified for kudu/127.11.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
I20260812 07:02:51.922356 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:35835
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:52.240226 18968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:52.240526 18968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:52.240680 18968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:52.247776 18968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:52.247902 18968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:52.248070 18968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:02:52.257496 18968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:35835
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:52.258656 18968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:52.259953 18968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:52.272739 18979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:52.272706 18983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:52.272666 18980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:52.273895 18968 server_base.cc:1034] running on GCE node
I20260812 07:02:52.274504 18968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:52.275385 18968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:52.276582 18968 hybrid_clock.cc:648] HybridClock initialized: now 1786518172276523 us; error 48 us; skew 500 ppm
I20260812 07:02:52.276880 18968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:52 dist-test-slave-lgr5 krb5kdc[18312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518172, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:52.279924 18968 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:02:52.281375 18968 webserver.cc:460] Webserver started at http://127.11.192.195:42719/ using document root <none> and password file <none>
I20260812 07:02:52.281994 18968 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:52.282101 18968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:52.282392 18968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:52.285300 18968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "2c01184f6c514fe4a6377242daed8afa"
format_stamp: "Formatted at 2026-08-12 07:02:52 on dist-test-slave-lgr5"
server_key: "79be30d836c14c09a3928207d9df1f05"
server_key_iv: "1e7171858f2b0b479d216bc789c9d492"
server_key_version: "encryptionkey@0"
I20260812 07:02:52.286073 18968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "2c01184f6c514fe4a6377242daed8afa"
format_stamp: "Formatted at 2026-08-12 07:02:52 on dist-test-slave-lgr5"
server_key: "79be30d836c14c09a3928207d9df1f05"
server_key_iv: "1e7171858f2b0b479d216bc789c9d492"
server_key_version: "encryptionkey@0"
I20260812 07:02:52.291109 18968 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 07:02:52.294625 18998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:52.295856 18968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:02:52.296011 18968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "2c01184f6c514fe4a6377242daed8afa"
format_stamp: "Formatted at 2026-08-12 07:02:52 on dist-test-slave-lgr5"
server_key: "79be30d836c14c09a3928207d9df1f05"
server_key_iv: "1e7171858f2b0b479d216bc789c9d492"
server_key_version: "encryptionkey@0"
I20260812 07:02:52.296238 18968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:52.313913 18968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:52.318040 18968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:52.318331 18968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:52.319248 18968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:02:52.320704 18968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:52.320793 18968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:52.320930 18968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:52.320989 18968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:52.354899 18968 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:35103
I20260812 07:02:52.354930 19217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:35103 every 8 connection(s)
I20260812 07:02:52.356248 18968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:02:52.358255 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 18968
I20260812 07:02:52.358397 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:02:52.358875 12035 external_mini_cluster.cc:1428] Setting key 53941af21ceb662389b8a82df3f5352f
Aug 12 07:02:52 dist-test-slave-lgr5 krb5kdc[18312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518172, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:52.375541 18715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:52.358851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:57469 (local address 127.11.192.254:35835)
0812 07:02:52.359071 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:52.359078 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:52.360459 (+ 1381us) server_negotiation.cc:407] Connection header received
0812 07:02:52.362371 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:52.362375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:52.362441 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:52.362525 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:52.363950 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:52.364736 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:52.366263 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:52.366660 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:52.366744 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:52.369931 (+ 3187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:52.369936 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:52.369939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:52.369970 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:52.372676 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:52.373568 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:52.373573 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:52.373577 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:52.373721 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:52.374193 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:52.374197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:52.374199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:52.374456 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:52.374571 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:52.375152 (+ 581us) server_negotiation.cc:299] Negotiation successful
0812 07:02:52.375302 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 07:02:52.376516 19220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:52.359348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:35835 (local address 127.11.192.195:57469)
0812 07:02:52.360287 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:52.360347 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:52.362131 (+ 1784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:52.362728 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:52.362735 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:52.363261 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:52.363817 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:52.363837 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:52.364891 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:52.364895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:52.366151 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:52.366165 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:52.366833 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:52.366839 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:52.367035 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:52.367595 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:52.367619 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:52.369735 (+ 2116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:52.372819 (+ 3084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:52.372828 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:52.372842 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:52.373449 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:52.373840 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:52.373843 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:52.373845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:52.374095 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:52.374562 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:52.374567 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:52.374862 (+ 295us) client_negotiation.cc:769] Sending connection context
0812 07:02:52.375108 (+ 246us) client_negotiation.cc:240] Negotiation successful
0812 07:02:52.375456 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":386,"threads_started":1}
I20260812 07:02:52.377696 19218 heartbeater.cc:347] Connected to a master server at 127.11.192.254:35835
I20260812 07:02:52.377969 19218 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:52.378636 19218 heartbeater.cc:511] Master 127.11.192.254:35835 requested a full tablet report, sending...
I20260812 07:02:52.379951 18391 ts_manager.cc:194] Registered new tserver with Master: 2c01184f6c514fe4a6377242daed8afa (127.11.192.195:35103)
I20260812 07:02:52.380721 18391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:57469
I20260812 07:02:52.385872 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 07:02:52 dist-test-slave-lgr5 krb5kdc[18312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260812 07:02:52.423244 18713 heartbeater.cc:503] Master 127.11.192.254:35835 was elected leader, sending a full tablet report...
Aug 12 07:02:52 dist-test-slave-lgr5 krb5kdc[18312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.193@KRBTEST.COM
I20260812 07:02:52.436627 19227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:52.421785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59828 (local address 127.11.192.193:37377)
0812 07:02:52.422407 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:52.422415 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:52.422441 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:02:52.422511 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:52.422517 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:52.422752 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:52.422897 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:52.425140 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:52.426135 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:52.427250 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:52.427785 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:52.427877 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:52.431357 (+ 3480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:52.431367 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:52.431380 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:52.431415 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:52.434007 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:52.434603 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:52.434610 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:52.434612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:52.434734 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:52.435092 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:52.435096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:52.435098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:52.435423 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:52.435715 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:52.436011 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 07:02:52.436289 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":336,"threads_started":1}
Aug 12 07:02:52 dist-test-slave-lgr5 krb5kdc[18312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:52.452499 18715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:52.440410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60542 (local address 127.11.192.254:35835)
0812 07:02:52.440607 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:52.440613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:52.440752 (+ 139us) server_negotiation.cc:407] Connection header received
0812 07:02:52.440895 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:52.440901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:52.440966 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:52.441048 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:52.442191 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:52.442905 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:52.444051 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:52.444518 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:52.444626 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:52.447905 (+ 3279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:52.447910 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:52.447913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:52.447943 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:52.450162 (+ 2219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:52.450712 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:52.450716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:52.450718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:52.450840 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:52.451188 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:52.451192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:52.451194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:52.451483 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:52.451716 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:52.451997 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 07:02:52.452222 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260812 07:02:52.453013 18391 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60542
I20260812 07:02:52.453898 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 18488
I20260812 07:02:52.465036 19227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:52.454629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59832 (local address 127.11.192.193:37377)
0812 07:02:52.454790 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:52.454796 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:52.455059 (+ 263us) server_negotiation.cc:407] Connection header received
0812 07:02:52.455191 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:52.455197 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:52.455256 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:52.455330 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:52.456442 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:52.457081 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:52.458205 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:52.458609 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:52.458716 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:52.460664 (+ 1948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:52.460669 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:52.460672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:52.460700 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:52.462772 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:52.463344 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:52.463350 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:52.463353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:52.463467 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:52.463805 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:52.463811 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:52.463814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:52.464039 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:52.464290 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:52.464579 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 07:02:52.464775 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 07:02:52.465629 18705 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:52.546720 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 18488
I20260812 07:02:52.573895 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 18720
Aug 12 07:02:52 dist-test-slave-lgr5 krb5kdc[18312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.194@KRBTEST.COM
I20260812 07:02:52.588552 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:52.574752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44630 (local address 127.11.192.194:38429)
0812 07:02:52.575352 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:52.575359 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:52.575386 (+ 27us) server_negotiation.cc:407] Connection header received
0812 07:02:52.575447 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:52.575453 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:52.575671 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:52.575801 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:52.576999 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:52.578050 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:52.579192 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:52.579712 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:52.579814 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:52.583624 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:52.583636 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:52.583650 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:52.583681 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:52.586368 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:52.586925 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:52.586932 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:52.586934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:52.587044 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:52.587311 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:52.587314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:52.587316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:52.587628 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:52.587853 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:52.588030 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 07:02:52.588260 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":313,"threads_started":1}
I20260812 07:02:52.589795 18952 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:52.669219 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 18720
I20260812 07:02:52.692654 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 18968
Aug 12 07:02:52 dist-test-slave-lgr5 krb5kdc[18312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.195@KRBTEST.COM
I20260812 07:02:52.708178 19242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:52.693601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60410 (local address 127.11.192.195:35103)
0812 07:02:52.694195 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:52.694202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:52.694230 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:02:52.694291 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:52.694297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:52.694518 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:52.694665 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:52.695985 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:52.697046 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:52.698399 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:52.698944 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:52.699071 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:52.703259 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:52.703271 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:52.703284 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:52.703319 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:52.705790 (+ 2471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:52.706378 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:52.706383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:52.706386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:52.706507 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:52.706811 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:52.706814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:52.706816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:52.707144 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:52.707374 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:52.707588 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 07:02:52.707781 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":282,"threads_started":1}
I20260812 07:02:52.709358 19212 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:52.789194 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 18968
I20260812 07:02:52.809593 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 18330
I20260812 07:02:52.810737 18446 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:52.913857 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 18330
2026-08-12T07:02:52Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3870 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:02:53 dist-test-slave-lgr5 krb5kdc[19247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:02:53 dist-test-slave-lgr5 krb5kdc[19247](info): set up 2 sockets
Aug 12 07:02:53 dist-test-slave-lgr5 krb5kdc[19247](info): commencing operation
krb5kdc: starting...
W20260812 07:02:54.157753 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.005s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:02:54 dist-test-slave-lgr5 krb5kdc[19247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:02:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:02:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:02:54.344476 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:40535
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:40535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:54.649758 19270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:54.650031 19270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:54.650170 19270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:54.657033 19270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:02:54.657136 19270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:54.657210 19270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:54.657284 19270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:02:54.657366 19270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:02:54.666540 19270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:40535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:40535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:54.667666 19270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:54.668917 19270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:54.679651 19284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:54.679736 19285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:54.679708 19290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:54.680475 19270 server_base.cc:1034] running on GCE node
I20260812 07:02:54.681170 19270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:54.682448 19270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:54.683645 19270 hybrid_clock.cc:648] HybridClock initialized: now 1786518174683604 us; error 36 us; skew 500 ppm
I20260812 07:02:54.683964 19270 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:54 dist-test-slave-lgr5 krb5kdc[19247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518174, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:54.690400 19270 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:02:54.691464 19270 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:54.691568 19270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:54.691875 19270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:54.694725 19270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "718b7f31008541a7a2512d539762c544"
format_stamp: "Formatted at 2026-08-12 07:02:54 on dist-test-slave-lgr5"
server_key: "b6060c5907a6222981134b4ce633a673"
server_key_iv: "b839c72ddeab49aeac783efa67827b42"
server_key_version: "encryptionkey@0"
I20260812 07:02:54.695492 19270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "718b7f31008541a7a2512d539762c544"
format_stamp: "Formatted at 2026-08-12 07:02:54 on dist-test-slave-lgr5"
server_key: "b6060c5907a6222981134b4ce633a673"
server_key_iv: "b839c72ddeab49aeac783efa67827b42"
server_key_version: "encryptionkey@0"
I20260812 07:02:54.700254 19270 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.003s
I20260812 07:02:54.703341 19300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:54.704521 19270 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:02:54.704671 19270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "718b7f31008541a7a2512d539762c544"
format_stamp: "Formatted at 2026-08-12 07:02:54 on dist-test-slave-lgr5"
server_key: "b6060c5907a6222981134b4ce633a673"
server_key_iv: "b839c72ddeab49aeac783efa67827b42"
server_key_version: "encryptionkey@0"
I20260812 07:02:54.704861 19270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:54.721307 19270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:54.724790 19270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:54.725023 19270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:54.740410 19270 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:40535
I20260812 07:02:54.740422 19389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:40535 every 8 connection(s)
I20260812 07:02:54.741199 19270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:02:54.744602 19392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:54.749606 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 19270
I20260812 07:02:54.749801 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:02:54.750579 12035 external_mini_cluster.cc:1428] Setting key 9c2c26732d8c0803ab396166cc198c59
I20260812 07:02:54.755324 19392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544: Bootstrap starting.
I20260812 07:02:54.758719 19392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:54.759935 19392 log.cc:824] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:54.762953 19392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544: No bootstrap required, opened a new log
Aug 12 07:02:54 dist-test-slave-lgr5 krb5kdc[19247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:54.770670 19392 raft_consensus.cc:348] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718b7f31008541a7a2512d539762c544" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40535 } }
I20260812 07:02:54.771023 19392 raft_consensus.cc:374] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:54.771137 19392 raft_consensus.cc:729] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 718b7f31008541a7a2512d539762c544, State: Initialized, Role: FOLLOWER
I20260812 07:02:54.771734 19392 consensus_queue.cc:260] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718b7f31008541a7a2512d539762c544" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40535 } }
I20260812 07:02:54.772022 19392 raft_consensus.cc:388] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:02:54.772182 19392 raft_consensus.cc:482] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:02:54.772374 19392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:54.773010 19395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:54.752559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51786 (local address 127.11.192.254:40535)
0812 07:02:54.753329 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:54.753341 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:54.753385 (+ 44us) server_negotiation.cc:407] Connection header received
0812 07:02:54.755008 (+ 1623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:54.755029 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:54.755394 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:54.755719 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:54.757545 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:54.758829 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:54.760584 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:54.761518 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:54.761653 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:54.766103 (+ 4450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:54.766111 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:54.766125 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:54.766161 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:54.769120 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:54.769766 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:54.769773 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:54.769782 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:54.769941 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:54.770304 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:54.770308 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:54.770310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:54.770822 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:54.771116 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:54.771574 (+ 458us) server_negotiation.cc:299] Negotiation successful
0812 07:02:54.771940 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":354,"threads_started":1}
I20260812 07:02:54.775036 19392 raft_consensus.cc:504] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718b7f31008541a7a2512d539762c544" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40535 } }
I20260812 07:02:54.775581 19392 leader_election.cc:302] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 718b7f31008541a7a2512d539762c544; no voters:
I20260812 07:02:54.776186 19392 leader_election.cc:288] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:02:54.776295 19398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:02:54.777276 19398 raft_consensus.cc:686] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544 [term 1 LEADER]: Becoming Leader. State: Replica: 718b7f31008541a7a2512d539762c544, State: Running, Role: LEADER
I20260812 07:02:54.777823 19398 consensus_queue.cc:237] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718b7f31008541a7a2512d539762c544" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40535 } }
I20260812 07:02:54.778187 19392 sys_catalog.cc:564] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:02:54.780575 19403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 718b7f31008541a7a2512d539762c544. Latest consensus state: current_term: 1 leader_uuid: "718b7f31008541a7a2512d539762c544" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718b7f31008541a7a2512d539762c544" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40535 } } }
I20260812 07:02:54.780437 19400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "718b7f31008541a7a2512d539762c544" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718b7f31008541a7a2512d539762c544" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40535 } } }
I20260812 07:02:54.781116 19400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544 [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:54.781121 19403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544 [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:54.783766 19416 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:02:54.788862 19416 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:02:54.796929 19416 catalog_manager.cc:1347] Generated new cluster ID: 2c625f3a01c64fb4a4a38909b45f6eba
I20260812 07:02:54.797031 19416 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:02:54.817991 19416 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:02:54.819228 19416 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:02:54.825977 19416 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544: Generated new TSK 0
I20260812 07:02:54.826802 19416 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
I20260812 07:02:54.885839 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:40535
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:55.199884 19442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:55.200166 19442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:55.200311 19442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:55.207216 19442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:55.207329 19442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:55.207477 19442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:02:55.216814 19442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:40535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:55.217957 19442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:55.219198 19442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:55.231220 19453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:55.231267 19459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:55.231320 19454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:55.232203 19442 server_base.cc:1034] running on GCE node
I20260812 07:02:55.232735 19442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:55.233640 19442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:55.234830 19442 hybrid_clock.cc:648] HybridClock initialized: now 1786518175234784 us; error 39 us; skew 500 ppm
I20260812 07:02:55.235128 19442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:55 dist-test-slave-lgr5 krb5kdc[19247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518175, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:55.238337 19442 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:02:55.239364 19442 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:55.239454 19442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:55.239711 19442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:55.242491 19442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "0bbe2b9734ed439fbbcd823fdfb0b7d3"
format_stamp: "Formatted at 2026-08-12 07:02:55 on dist-test-slave-lgr5"
server_key: "f4861eb3d8257894d11111b904ffaec3"
server_key_iv: "adc915a192053fdedf6577e2c3d200c0"
server_key_version: "encryptionkey@0"
I20260812 07:02:55.243274 19442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "0bbe2b9734ed439fbbcd823fdfb0b7d3"
format_stamp: "Formatted at 2026-08-12 07:02:55 on dist-test-slave-lgr5"
server_key: "f4861eb3d8257894d11111b904ffaec3"
server_key_iv: "adc915a192053fdedf6577e2c3d200c0"
server_key_version: "encryptionkey@0"
I20260812 07:02:55.247923 19442 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 07:02:55.251098 19469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:55.252322 19442 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:02:55.252472 19442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "0bbe2b9734ed439fbbcd823fdfb0b7d3"
format_stamp: "Formatted at 2026-08-12 07:02:55 on dist-test-slave-lgr5"
server_key: "f4861eb3d8257894d11111b904ffaec3"
server_key_iv: "adc915a192053fdedf6577e2c3d200c0"
server_key_version: "encryptionkey@0"
I20260812 07:02:55.252666 19442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:55.267939 19442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:55.271319 19442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:55.271538 19442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:55.272475 19442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:02:55.273789 19442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:55.273875 19442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:55.273952 19442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:55.274010 19442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:55.303355 19442 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:36355
I20260812 07:02:55.303413 19664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:36355 every 8 connection(s)
I20260812 07:02:55.304150 19442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:02:55.314204 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 19442
I20260812 07:02:55.314373 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:02:55.315037 12035 external_mini_cluster.cc:1428] Setting key deac3499f20f52befb3b3b932ed584e9
Aug 12 07:02:55 dist-test-slave-lgr5 krb5kdc[19247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518175, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:55.322605 19667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:55.306398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:55515 (local address 127.11.192.254:40535)
0812 07:02:55.306840 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:55.306846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:55.307763 (+ 917us) server_negotiation.cc:407] Connection header received
0812 07:02:55.309622 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:55.309626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:55.309691 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:55.309773 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:55.311203 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:55.311909 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:55.313371 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:55.313751 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:55.313873 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:55.317330 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:55.317335 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:55.317338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:55.317367 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:55.319729 (+ 2362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:55.320466 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:55.320470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:55.320472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:55.320596 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:55.320987 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:55.320990 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:55.320993 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:55.321256 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:55.321387 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:55.322018 (+ 631us) server_negotiation.cc:299] Negotiation successful
0812 07:02:55.322246 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":217,"threads_started":1}
I20260812 07:02:55.323397 19668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:55.306915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:40535 (local address 127.11.192.193:55515)
0812 07:02:55.307609 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:55.307647 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:55.309362 (+ 1715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:55.309947 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:55.309954 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:55.310433 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:55.311040 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:55.311066 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:55.312023 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:55.312026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:55.313261 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:55.313275 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:55.313921 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:55.313926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:55.314093 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:55.314727 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:55.314748 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:55.317133 (+ 2385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:55.319868 (+ 2735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:55.319877 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:55.319891 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:55.320349 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:55.320698 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:55.320703 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:55.320707 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:55.320875 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:55.321381 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:55.321388 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:55.321678 (+ 290us) client_negotiation.cc:769] Sending connection context
0812 07:02:55.321894 (+ 216us) client_negotiation.cc:240] Negotiation successful
0812 07:02:55.322366 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":268,"threads_started":1}
I20260812 07:02:55.324745 19665 heartbeater.cc:347] Connected to a master server at 127.11.192.254:40535
I20260812 07:02:55.325043 19665 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:55.325775 19665 heartbeater.cc:511] Master 127.11.192.254:40535 requested a full tablet report, sending...
I20260812 07:02:55.328025 19335 ts_manager.cc:194] Registered new tserver with Master: 0bbe2b9734ed439fbbcd823fdfb0b7d3 (127.11.192.193:36355)
I20260812 07:02:55.329947 19335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:55515
WARNING: no policy specified for kudu/127.11.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
I20260812 07:02:55.360610 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:40535
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:55.668149 19672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:55.668440 19672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:55.668591 19672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:55.675547 19672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:55.675638 19672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:55.675784 19672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:02:55.685240 19672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:40535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:55.686307 19672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:55.687454 19672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:55.705251 19682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:55.705219 19684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:55.705960 19688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:55.706209 19672 server_base.cc:1034] running on GCE node
I20260812 07:02:55.706920 19672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:55.708098 19672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:55.709362 19672 hybrid_clock.cc:648] HybridClock initialized: now 1786518175709288 us; error 39 us; skew 500 ppm
I20260812 07:02:55.709839 19672 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:55 dist-test-slave-lgr5 krb5kdc[19247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518175, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:55.714375 19672 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:02:55.715682 19672 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:55.715802 19672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:55.716190 19672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:55.719583 19672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "1da1499f820b483e850f6f8ab05296ee"
format_stamp: "Formatted at 2026-08-12 07:02:55 on dist-test-slave-lgr5"
server_key: "54284f727d4d86a40e2a977b19007e3e"
server_key_iv: "793e05cf4ed2d9926a1766d0156167b2"
server_key_version: "encryptionkey@0"
I20260812 07:02:55.720639 19672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "1da1499f820b483e850f6f8ab05296ee"
format_stamp: "Formatted at 2026-08-12 07:02:55 on dist-test-slave-lgr5"
server_key: "54284f727d4d86a40e2a977b19007e3e"
server_key_iv: "793e05cf4ed2d9926a1766d0156167b2"
server_key_version: "encryptionkey@0"
I20260812 07:02:55.726752 19672 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.002s
I20260812 07:02:55.731149 19699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:55.733129 19672 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 07:02:55.733278 19672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "1da1499f820b483e850f6f8ab05296ee"
format_stamp: "Formatted at 2026-08-12 07:02:55 on dist-test-slave-lgr5"
server_key: "54284f727d4d86a40e2a977b19007e3e"
server_key_iv: "793e05cf4ed2d9926a1766d0156167b2"
server_key_version: "encryptionkey@0"
I20260812 07:02:55.733481 19672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:55.753756 19672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:55.757521 19672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:55.757725 19672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:55.758530 19672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:02:55.759699 19672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:55.759780 19672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:55.759858 19672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:55.759912 19672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:55.788863 19672 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:41085
I20260812 07:02:55.788910 19888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:41085 every 8 connection(s)
I20260812 07:02:55.789608 19672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:02:55.795890 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 19672
I20260812 07:02:55.796042 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:02:55.796463 12035 external_mini_cluster.cc:1428] Setting key 7e0265585767ac8e2400bd51332a5414
Aug 12 07:02:55 dist-test-slave-lgr5 krb5kdc[19247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518175, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:55.808394 19667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:55.791760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:37161 (local address 127.11.192.254:40535)
0812 07:02:55.791923 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:55.791928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:55.793028 (+ 1100us) server_negotiation.cc:407] Connection header received
0812 07:02:55.794702 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:55.794707 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:55.794774 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:55.794858 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:55.796282 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:55.797112 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:55.799051 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:55.799561 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:55.799658 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:55.803212 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:55.803217 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:55.803221 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:55.803251 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:55.805586 (+ 2335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:55.806246 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:55.806250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:55.806252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:55.806375 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:55.806763 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:55.806766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:55.806768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:55.807010 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:55.807149 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:55.807819 (+ 670us) server_negotiation.cc:299] Negotiation successful
0812 07:02:55.808045 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 07:02:55.809242 19893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:55.792217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:40535 (local address 127.11.192.194:37161)
0812 07:02:55.792872 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:55.792907 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:55.794472 (+ 1565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:55.795032 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:55.795051 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:55.795528 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:55.796096 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:55.796124 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:55.797272 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:55.797277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:55.798856 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:55.798875 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:55.799783 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:55.799790 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:55.800055 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:55.800725 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:55.800752 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:55.803020 (+ 2268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:55.805715 (+ 2695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:55.805724 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:55.805737 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:55.806130 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:55.806477 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:55.806482 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:55.806486 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:55.806652 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:55.807139 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:55.807146 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:55.807440 (+ 294us) client_negotiation.cc:769] Sending connection context
0812 07:02:55.807697 (+ 257us) client_negotiation.cc:240] Negotiation successful
0812 07:02:55.808221 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":255,"threads_started":1}
I20260812 07:02:55.810528 19889 heartbeater.cc:347] Connected to a master server at 127.11.192.254:40535
I20260812 07:02:55.810783 19889 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:55.811360 19889 heartbeater.cc:511] Master 127.11.192.254:40535 requested a full tablet report, sending...
I20260812 07:02:55.812669 19335 ts_manager.cc:194] Registered new tserver with Master: 1da1499f820b483e850f6f8ab05296ee (127.11.192.194:41085)
I20260812 07:02:55.813416 19335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:37161
WARNING: no policy specified for kudu/127.11.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
I20260812 07:02:55.841385 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:40535
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:02:56.153107 19897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:56.153363 19897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:56.153502 19897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:56.160614 19897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:56.160708 19897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:56.160866 19897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:02:56.169991 19897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:40535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:56.171105 19897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:56.172330 19897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:56.184146 19908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:56.184204 19913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:56.184458 19917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:56.185142 19897 server_base.cc:1034] running on GCE node
I20260812 07:02:56.185707 19897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:56.186510 19897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:56.187695 19897 hybrid_clock.cc:648] HybridClock initialized: now 1786518176187646 us; error 44 us; skew 500 ppm
I20260812 07:02:56.187974 19897 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:02:56 dist-test-slave-lgr5 krb5kdc[19247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518176, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:02:56.190868 19897 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:02:56.191938 19897 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:56.192035 19897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:56.192368 19897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:56.195115 19897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "d7dced86cb77472bb2606fb6c1fc64db"
format_stamp: "Formatted at 2026-08-12 07:02:56 on dist-test-slave-lgr5"
server_key: "ad8f2b63c247c1cf8c2765ecd8b92914"
server_key_iv: "6571a33fd5231a3bc537549eb31c4a24"
server_key_version: "encryptionkey@0"
I20260812 07:02:56.195896 19897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "d7dced86cb77472bb2606fb6c1fc64db"
format_stamp: "Formatted at 2026-08-12 07:02:56 on dist-test-slave-lgr5"
server_key: "ad8f2b63c247c1cf8c2765ecd8b92914"
server_key_iv: "6571a33fd5231a3bc537549eb31c4a24"
server_key_version: "encryptionkey@0"
I20260812 07:02:56.200577 19897 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:02:56.203719 19928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:56.204934 19897 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:02:56.205088 19897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "d7dced86cb77472bb2606fb6c1fc64db"
format_stamp: "Formatted at 2026-08-12 07:02:56 on dist-test-slave-lgr5"
server_key: "ad8f2b63c247c1cf8c2765ecd8b92914"
server_key_iv: "6571a33fd5231a3bc537549eb31c4a24"
server_key_version: "encryptionkey@0"
I20260812 07:02:56.205273 19897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:56.234797 19897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:56.238773 19897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:56.239006 19897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:56.239857 19897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:02:56.241122 19897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:56.241209 19897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:56.241312 19897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:56.241376 19897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:56.270781 19897 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:41891
I20260812 07:02:56.270840 20121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:41891 every 8 connection(s)
I20260812 07:02:56.271534 19897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:02:56.277956 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 19897
I20260812 07:02:56.278122 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:02:56.278554 12035 external_mini_cluster.cc:1428] Setting key 87a50149e86debe5a60d4fc6f293033e
Aug 12 07:02:56 dist-test-slave-lgr5 krb5kdc[19247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518176, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:02:56.290148 19667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.273728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:48185 (local address 127.11.192.254:40535)
0812 07:02:56.273890 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:56.273895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:56.274998 (+ 1103us) server_negotiation.cc:407] Connection header received
0812 07:02:56.276974 (+ 1976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:56.276985 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:56.277073 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:56.277161 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:56.278664 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.279550 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.281349 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.281780 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.281875 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:56.285081 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:56.285086 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:56.285089 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:56.285120 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:56.287355 (+ 2235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:56.288016 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:56.288021 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:56.288024 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:56.288182 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:56.288574 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:56.288580 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:56.288583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:56.288842 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:56.288973 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:56.289619 (+ 646us) server_negotiation.cc:299] Negotiation successful
0812 07:02:56.289828 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:02:56.291008 20134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.274184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:40535 (local address 127.11.192.195:48185)
0812 07:02:56.274846 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:56.274891 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:56.276713 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:56.277397 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:56.277405 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:56.277909 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:56.278495 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:56.278513 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.279710 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.279716 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:56.281211 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:56.281226 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.281967 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.281974 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:56.282153 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:56.282731 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:02:56.282758 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:56.284881 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:56.287484 (+ 2603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:56.287493 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:56.287506 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:56.287891 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:56.288282 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:02:56.288287 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:02:56.288291 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:02:56.288456 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:02:56.288965 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:56.288972 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:56.289265 (+ 293us) client_negotiation.cc:769] Sending connection context
0812 07:02:56.289490 (+ 225us) client_negotiation.cc:240] Negotiation successful
0812 07:02:56.289935 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":272,"threads_started":1}
I20260812 07:02:56.292218 20123 heartbeater.cc:347] Connected to a master server at 127.11.192.254:40535
I20260812 07:02:56.292483 20123 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:56.293121 20123 heartbeater.cc:511] Master 127.11.192.254:40535 requested a full tablet report, sending...
I20260812 07:02:56.294412 19335 ts_manager.cc:194] Registered new tserver with Master: d7dced86cb77472bb2606fb6c1fc64db (127.11.192.195:41891)
I20260812 07:02:56.295190 19335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:48185
I20260812 07:02:56.305207 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:02:56.321830 19667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.309512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51792 (local address 127.11.192.254:40535)
0812 07:02:56.309660 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:56.309664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:56.310231 (+ 567us) server_negotiation.cc:407] Connection header received
0812 07:02:56.310379 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:56.310385 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:56.310448 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:56.310527 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:56.311898 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.312616 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.313841 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.314256 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.314361 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:56.316703 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:56.316709 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:56.316713 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:56.316750 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:56.319205 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:56.319855 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:56.319862 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:56.319866 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:56.320002 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:56.320397 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:56.320403 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:56.320405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:56.320710 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:56.320947 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:56.321328 (+ 381us) server_negotiation.cc:299] Negotiation successful
0812 07:02:56.321532 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:02:56.325587 19335 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 07:02:56.332710 19665 heartbeater.cc:503] Master 127.11.192.254:40535 was elected leader, sending a full tablet report...
W20260812 07:02:56.333555 19335 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:02:56.356613 20160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.344939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:41085 (local address 127.0.0.1:50386)
0812 07:02:56.346803 (+ 1864us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:56.346822 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:56.346941 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:56.347945 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:56.347951 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:56.347982 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:56.348500 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:56.348513 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.350569 (+ 2056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.350575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:56.353082 (+ 2507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:56.353102 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.355466 (+ 2364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.355475 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:56.355720 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:56.355776 (+ 56us) client_negotiation.cc:769] Sending connection context
0812 07:02:56.355927 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 07:02:56.356140 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1727,"thread_start_us":1589,"threads_started":1}
I20260812 07:02:56.357278 20153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.344994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52130 (local address 127.11.192.193:36355)
0812 07:02:56.345610 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:56.345618 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:56.346355 (+ 737us) server_negotiation.cc:407] Connection header received
0812 07:02:56.346722 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:56.346728 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:56.346922 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:56.347108 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:56.348029 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.349743 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.352974 (+ 3231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.354527 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.354706 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:56.354876 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:56.356724 (+ 1848us) server_negotiation.cc:299] Negotiation successful
0812 07:02:56.356924 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":314,"threads_started":1}
I20260812 07:02:56.357573 20157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.345410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50386 (local address 127.11.192.194:41085)
0812 07:02:56.345909 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:56.345918 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:56.347084 (+ 1166us) server_negotiation.cc:407] Connection header received
0812 07:02:56.347219 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:56.347228 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:56.347486 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:56.347689 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:56.348741 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.350385 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.353323 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.354904 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.355111 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:56.355302 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:56.357002 (+ 1700us) server_negotiation.cc:299] Negotiation successful
0812 07:02:56.357193 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":309,"threads_started":1}
I20260812 07:02:56.358258 20158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.344908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:36355 (local address 127.0.0.1:52130)
0812 07:02:56.346230 (+ 1322us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:56.346262 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:56.346506 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:56.347378 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:56.347384 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:56.347414 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:56.347849 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:56.347862 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.350093 (+ 2231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.350099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:56.352782 (+ 2683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:56.352800 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.356282 (+ 3482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.356294 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:56.356501 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:56.356541 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 07:02:56.357763 (+ 1222us) client_negotiation.cc:240] Negotiation successful
0812 07:02:56.357911 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1118,"thread_start_us":963,"threads_started":1}
I20260812 07:02:56.362288 19549 tablet_service.cc:1467] Processing CreateTablet for tablet 4a8bf54df12c45beb69455bac0b5f774 (DEFAULT_TABLE table=test-table [id=b9d71bbdfa0a44fe93433e1153858a40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:56.363776 19549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a8bf54df12c45beb69455bac0b5f774. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:56.363385 19777 tablet_service.cc:1467] Processing CreateTablet for tablet 4a8bf54df12c45beb69455bac0b5f774 (DEFAULT_TABLE table=test-table [id=b9d71bbdfa0a44fe93433e1153858a40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:56.364781 19777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a8bf54df12c45beb69455bac0b5f774. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:56.365541 20160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.346941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:41891 (local address 127.0.0.1:40764)
0812 07:02:56.357981 (+ 11040us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:56.358003 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:56.358116 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:56.360068 (+ 1952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:56.360073 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:56.360126 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:56.360309 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:56.360323 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.362051 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.362057 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:56.363325 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:56.363338 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.365009 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.365016 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:56.365175 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:56.365201 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 07:02:56.365261 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 07:02:56.365360 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10939,"thread_start_us":12869,"threads_started":1}
I20260812 07:02:56.366375 20164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.347316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40764 (local address 127.11.192.195:41891)
0812 07:02:56.359064 (+ 11748us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:56.359074 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:56.359112 (+ 38us) server_negotiation.cc:407] Connection header received
0812 07:02:56.359228 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:56.359236 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:56.359466 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:56.360506 (+ 1040us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:56.360826 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.361910 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.363493 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.364827 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.365844 (+ 1017us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:56.365958 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:56.366019 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 07:02:56.366122 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11555,"thread_start_us":11367,"threads_started":1}
I20260812 07:02:56.369237 20018 tablet_service.cc:1467] Processing CreateTablet for tablet 4a8bf54df12c45beb69455bac0b5f774 (DEFAULT_TABLE table=test-table [id=b9d71bbdfa0a44fe93433e1153858a40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:56.370522 20018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a8bf54df12c45beb69455bac0b5f774. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:56.378854 20166 tablet_bootstrap.cc:492] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3: Bootstrap starting.
I20260812 07:02:56.380622 20168 tablet_bootstrap.cc:492] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee: Bootstrap starting.
I20260812 07:02:56.382681 20166 tablet_bootstrap.cc:654] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:56.382764 20169 tablet_bootstrap.cc:492] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db: Bootstrap starting.
I20260812 07:02:56.383713 20168 tablet_bootstrap.cc:654] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:56.383967 20166 log.cc:824] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:56.384923 20168 log.cc:824] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:56.386128 20169 tablet_bootstrap.cc:654] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:56.387398 20169 log.cc:824] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:56.392944 20169 tablet_bootstrap.cc:492] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db: No bootstrap required, opened a new log
I20260812 07:02:56.393024 20168 tablet_bootstrap.cc:492] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee: No bootstrap required, opened a new log
I20260812 07:02:56.393036 20166 tablet_bootstrap.cc:492] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3: No bootstrap required, opened a new log
I20260812 07:02:56.393187 20169 ts_tablet_manager.cc:1397] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260812 07:02:56.393280 20168 ts_tablet_manager.cc:1397] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.010s
I20260812 07:02:56.393287 20166 ts_tablet_manager.cc:1397] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.004s
I20260812 07:02:56.400170 20169 raft_consensus.cc:348] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bbe2b9734ed439fbbcd823fdfb0b7d3" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 36355 } } peers { permanent_uuid: "1da1499f820b483e850f6f8ab05296ee" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 41085 } } peers { permanent_uuid: "d7dced86cb77472bb2606fb6c1fc64db" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41891 } }
I20260812 07:02:56.400506 20169 raft_consensus.cc:374] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:56.400602 20169 raft_consensus.cc:729] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7dced86cb77472bb2606fb6c1fc64db, State: Initialized, Role: FOLLOWER
I20260812 07:02:56.401154 20169 consensus_queue.cc:260] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bbe2b9734ed439fbbcd823fdfb0b7d3" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 36355 } } peers { permanent_uuid: "1da1499f820b483e850f6f8ab05296ee" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 41085 } } peers { permanent_uuid: "d7dced86cb77472bb2606fb6c1fc64db" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41891 } }
I20260812 07:02:56.402191 20123 heartbeater.cc:503] Master 127.11.192.254:40535 was elected leader, sending a full tablet report...
I20260812 07:02:56.402644 20169 ts_tablet_manager.cc:1428] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db: Time spent starting tablet: real 0.009s user 0.010s sys 0.000s
I20260812 07:02:56.403203 20166 raft_consensus.cc:348] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bbe2b9734ed439fbbcd823fdfb0b7d3" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 36355 } } peers { permanent_uuid: "1da1499f820b483e850f6f8ab05296ee" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 41085 } } peers { permanent_uuid: "d7dced86cb77472bb2606fb6c1fc64db" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41891 } }
I20260812 07:02:56.403606 20166 raft_consensus.cc:374] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:56.403753 20166 raft_consensus.cc:729] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bbe2b9734ed439fbbcd823fdfb0b7d3, State: Initialized, Role: FOLLOWER
I20260812 07:02:56.404027 20168 raft_consensus.cc:348] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bbe2b9734ed439fbbcd823fdfb0b7d3" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 36355 } } peers { permanent_uuid: "1da1499f820b483e850f6f8ab05296ee" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 41085 } } peers { permanent_uuid: "d7dced86cb77472bb2606fb6c1fc64db" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41891 } }
I20260812 07:02:56.404459 20168 raft_consensus.cc:374] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:56.404397 20166 consensus_queue.cc:260] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bbe2b9734ed439fbbcd823fdfb0b7d3" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 36355 } } peers { permanent_uuid: "1da1499f820b483e850f6f8ab05296ee" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 41085 } } peers { permanent_uuid: "d7dced86cb77472bb2606fb6c1fc64db" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41891 } }
I20260812 07:02:56.404580 20168 raft_consensus.cc:729] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1da1499f820b483e850f6f8ab05296ee, State: Initialized, Role: FOLLOWER
I20260812 07:02:56.405144 20168 consensus_queue.cc:260] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bbe2b9734ed439fbbcd823fdfb0b7d3" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 36355 } } peers { permanent_uuid: "1da1499f820b483e850f6f8ab05296ee" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 41085 } } peers { permanent_uuid: "d7dced86cb77472bb2606fb6c1fc64db" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41891 } }
I20260812 07:02:56.406288 20166 ts_tablet_manager.cc:1428] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260812 07:02:56.408454 19889 heartbeater.cc:503] Master 127.11.192.254:40535 was elected leader, sending a full tablet report...
I20260812 07:02:56.410243 20168 ts_tablet_manager.cc:1428] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee: Time spent starting tablet: real 0.017s user 0.015s sys 0.001s
I20260812 07:02:56.478654 20180 raft_consensus.cc:482] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:02:56.478971 20180 raft_consensus.cc:504] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bbe2b9734ed439fbbcd823fdfb0b7d3" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 36355 } } peers { permanent_uuid: "1da1499f820b483e850f6f8ab05296ee" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 41085 } } peers { permanent_uuid: "d7dced86cb77472bb2606fb6c1fc64db" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41891 } }
I20260812 07:02:56.480712 20180 leader_election.cc:288] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bbe2b9734ed439fbbcd823fdfb0b7d3 (127.11.192.193:36355), d7dced86cb77472bb2606fb6c1fc64db (127.11.192.195:41891)
I20260812 07:02:56.487490 20177 raft_consensus.cc:482] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:02:56.487777 20177 raft_consensus.cc:504] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bbe2b9734ed439fbbcd823fdfb0b7d3" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 36355 } } peers { permanent_uuid: "1da1499f820b483e850f6f8ab05296ee" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 41085 } } peers { permanent_uuid: "d7dced86cb77472bb2606fb6c1fc64db" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41891 } }
I20260812 07:02:56.489410 20177 leader_election.cc:288] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bbe2b9734ed439fbbcd823fdfb0b7d3 (127.11.192.193:36355), 1da1499f820b483e850f6f8ab05296ee (127.11.192.194:41085)
I20260812 07:02:56.490563 20164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.481146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:35377 (local address 127.11.192.195:41891)
0812 07:02:56.481344 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:56.481351 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:56.482262 (+ 911us) server_negotiation.cc:407] Connection header received
0812 07:02:56.482451 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:56.482457 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:56.482535 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:56.482631 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:56.483475 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.484599 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.487773 (+ 3174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.489155 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.489290 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:56.489365 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:56.490162 (+ 797us) server_negotiation.cc:299] Negotiation successful
0812 07:02:56.490299 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:02:56.490581 20153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.481263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:40873 (local address 127.11.192.193:36355)
0812 07:02:56.481435 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:56.481441 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:56.482209 (+ 768us) server_negotiation.cc:407] Connection header received
0812 07:02:56.482452 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:56.482457 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:56.482545 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:56.482631 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:56.483429 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.484599 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.487764 (+ 3165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.489074 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.489220 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:56.489327 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:56.490143 (+ 816us) server_negotiation.cc:299] Negotiation successful
0812 07:02:56.490304 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:02:56.491501 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.481053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:41891 (local address 127.11.192.194:35377)
0812 07:02:56.482160 (+ 1107us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:56.482176 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:56.482332 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:56.482656 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:56.482662 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:56.482693 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:56.483289 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:56.483301 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.484737 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.484744 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:56.487559 (+ 2815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:56.487569 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.489595 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.489600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:56.489896 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:56.489964 (+ 68us) client_negotiation.cc:769] Sending connection context
0812 07:02:56.490898 (+ 934us) client_negotiation.cc:240] Negotiation successful
0812 07:02:56.491041 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"mutex_wait_us":295,"spinlock_wait_cycles":23296,"thread_start_us":508,"threads_started":1}
I20260812 07:02:56.491814 20183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.480946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:36355 (local address 127.11.192.194:40873)
0812 07:02:56.482083 (+ 1137us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:56.482106 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:56.482307 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:56.482656 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:56.482662 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:56.482719 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:56.483232 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:56.483253 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.484743 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.484748 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:56.487549 (+ 2801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:56.487569 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.489579 (+ 2010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.489587 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:56.489889 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:56.489957 (+ 68us) client_negotiation.cc:769] Sending connection context
0812 07:02:56.491184 (+ 1227us) client_negotiation.cc:240] Negotiation successful
0812 07:02:56.491322 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":953,"mutex_wait_us":238,"spinlock_wait_cycles":34432,"thread_start_us":771,"threads_started":1}
I20260812 07:02:56.492880 20052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a8bf54df12c45beb69455bac0b5f774" candidate_uuid: "1da1499f820b483e850f6f8ab05296ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7dced86cb77472bb2606fb6c1fc64db" is_pre_election: true
I20260812 07:02:56.492921 19586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a8bf54df12c45beb69455bac0b5f774" candidate_uuid: "1da1499f820b483e850f6f8ab05296ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bbe2b9734ed439fbbcd823fdfb0b7d3" is_pre_election: true
I20260812 07:02:56.493428 20052 raft_consensus.cc:2454] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1da1499f820b483e850f6f8ab05296ee in term 0.
I20260812 07:02:56.493512 19586 raft_consensus.cc:2454] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1da1499f820b483e850f6f8ab05296ee in term 0.
I20260812 07:02:56.494284 19702 leader_election.cc:302] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0bbe2b9734ed439fbbcd823fdfb0b7d3, 1da1499f820b483e850f6f8ab05296ee, d7dced86cb77472bb2606fb6c1fc64db; no voters:
I20260812 07:02:56.494745 20180 raft_consensus.cc:2781] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:02:56.494921 20180 raft_consensus.cc:482] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:02:56.495034 20180 raft_consensus.cc:3037] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:56.498354 20180 raft_consensus.cc:504] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bbe2b9734ed439fbbcd823fdfb0b7d3" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 36355 } } peers { permanent_uuid: "1da1499f820b483e850f6f8ab05296ee" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 41085 } } peers { permanent_uuid: "d7dced86cb77472bb2606fb6c1fc64db" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41891 } }
I20260812 07:02:56.499120 20180 leader_election.cc:288] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee [CANDIDATE]: Term 1 election: Requested vote from peers 0bbe2b9734ed439fbbcd823fdfb0b7d3 (127.11.192.193:36355), d7dced86cb77472bb2606fb6c1fc64db (127.11.192.195:41891)
I20260812 07:02:56.499985 20134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.489584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:36355 (local address 127.11.192.195:39909)
0812 07:02:56.489908 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:56.489925 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:56.490019 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:56.491090 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:56.491096 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:56.491128 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:56.491320 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:56.491329 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.495251 (+ 3922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.495257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:56.497229 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:56.497247 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.499165 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.499172 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:56.499425 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:56.499467 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 07:02:56.499546 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 07:02:56.499675 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198}
I20260812 07:02:56.500154 20052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a8bf54df12c45beb69455bac0b5f774" candidate_uuid: "1da1499f820b483e850f6f8ab05296ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7dced86cb77472bb2606fb6c1fc64db"
I20260812 07:02:56.500396 20052 raft_consensus.cc:3037] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:56.501883 20153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.490226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:39909 (local address 127.11.192.193:36355)
0812 07:02:56.490805 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:56.490811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:56.490830 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:02:56.490892 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:56.490897 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:56.490971 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:56.491062 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:56.492282 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.494816 (+ 2534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.497710 (+ 2894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.498950 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.501367 (+ 2417us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:56.501438 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:56.501500 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 07:02:56.501616 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":10537,"threads_started":1}
I20260812 07:02:56.501878 19586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a8bf54df12c45beb69455bac0b5f774" candidate_uuid: "1da1499f820b483e850f6f8ab05296ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bbe2b9734ed439fbbcd823fdfb0b7d3"
I20260812 07:02:56.502174 19586 raft_consensus.cc:3037] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:56.503509 20052 raft_consensus.cc:2454] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1da1499f820b483e850f6f8ab05296ee in term 1.
I20260812 07:02:56.503898 19700 leader_election.cc:302] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1da1499f820b483e850f6f8ab05296ee, d7dced86cb77472bb2606fb6c1fc64db; no voters:
I20260812 07:02:56.504215 20180 raft_consensus.cc:2781] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:02:56.504684 20180 raft_consensus.cc:686] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee [term 1 LEADER]: Becoming Leader. State: Replica: 1da1499f820b483e850f6f8ab05296ee, State: Running, Role: LEADER
I20260812 07:02:56.505291 20180 consensus_queue.cc:237] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bbe2b9734ed439fbbcd823fdfb0b7d3" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 36355 } } peers { permanent_uuid: "1da1499f820b483e850f6f8ab05296ee" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 41085 } } peers { permanent_uuid: "d7dced86cb77472bb2606fb6c1fc64db" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41891 } }
I20260812 07:02:56.505496 19586 raft_consensus.cc:2454] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1da1499f820b483e850f6f8ab05296ee in term 1.
I20260812 07:02:56.505918 19580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a8bf54df12c45beb69455bac0b5f774" candidate_uuid: "d7dced86cb77472bb2606fb6c1fc64db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bbe2b9734ed439fbbcd823fdfb0b7d3" is_pre_election: true
I20260812 07:02:56.506331 19580 raft_consensus.cc:2379] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d7dced86cb77472bb2606fb6c1fc64db in current term 1: Already voted for candidate 1da1499f820b483e850f6f8ab05296ee in this term.
I20260812 07:02:56.507273 20157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.490537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:48363 (local address 127.11.192.194:41085)
0812 07:02:56.490714 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:56.490720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:56.500326 (+ 9606us) server_negotiation.cc:407] Connection header received
0812 07:02:56.500648 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:56.500654 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:56.500731 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:56.500816 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:02:56.501500 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.502517 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.504690 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.505974 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.506112 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:56.506178 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:56.506918 (+ 740us) server_negotiation.cc:299] Negotiation successful
0812 07:02:56.507037 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:02:56.507402 20134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.489963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:41085 (local address 127.11.192.195:48363)
0812 07:02:56.500231 (+ 10268us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:56.500251 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:56.500433 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:56.501022 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:56.501027 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:56.501058 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:02:56.501315 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:56.501328 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.502684 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.502690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:56.504475 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:02:56.504490 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.506503 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.506515 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:02:56.506720 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:56.506753 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:02:56.506840 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 07:02:56.507015 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10175,"thread_start_us":11008,"threads_started":1}
I20260812 07:02:56.508513 19815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a8bf54df12c45beb69455bac0b5f774" candidate_uuid: "d7dced86cb77472bb2606fb6c1fc64db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1da1499f820b483e850f6f8ab05296ee" is_pre_election: true
I20260812 07:02:56.509696 19929 leader_election.cc:302] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7dced86cb77472bb2606fb6c1fc64db; no voters: 0bbe2b9734ed439fbbcd823fdfb0b7d3, 1da1499f820b483e850f6f8ab05296ee
I20260812 07:02:56.510138 20177 raft_consensus.cc:2726] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 07:02:56.510977 19332 catalog_manager.cc:5495] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 1da1499f820b483e850f6f8ab05296ee (127.11.192.194). New cstate: current_term: 1 leader_uuid: "1da1499f820b483e850f6f8ab05296ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bbe2b9734ed439fbbcd823fdfb0b7d3" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 36355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1da1499f820b483e850f6f8ab05296ee" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 41085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7dced86cb77472bb2606fb6c1fc64db" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41891 } health_report { overall_health: UNKNOWN } } }
W20260812 07:02:56.522856 20129 tablet.cc:2369] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:02:56.541232 19890 tablet.cc:2369] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:02:56.547001 20157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.541923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50390 (local address 127.11.192.194:41085)
0812 07:02:56.542070 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:56.542076 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:56.542154 (+ 78us) server_negotiation.cc:407] Connection header received
0812 07:02:56.542358 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:56.542362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:56.542444 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:56.542573 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:02:56.542923 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.543650 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.544994 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.545386 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.545503 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:56.545936 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:02:56.546423 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:02:56.546563 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:56.546662 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 07:02:56.546739 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 07:02:56.554454 20052 raft_consensus.cc:1261] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db [term 1 FOLLOWER]: Refusing update from remote peer 1da1499f820b483e850f6f8ab05296ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:56.554509 19580 raft_consensus.cc:1261] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3 [term 1 FOLLOWER]: Refusing update from remote peer 1da1499f820b483e850f6f8ab05296ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:56.555272 20188 consensus_queue.cc:1035] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7dced86cb77472bb2606fb6c1fc64db" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:02:56.555565 20180 consensus_queue.cc:1035] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bbe2b9734ed439fbbcd823fdfb0b7d3" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 36355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 07:02:56.556176 19666 tablet.cc:2369] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:02:56.567755 20193 mvcc.cc:205] Tried to move back new op lower bound from 7317578451158233088 to 7317578450970595328. Current Snapshot: MvccSnapshot[applied={T|T < 7317578451158233088}]
I20260812 07:02:56.580237 19332 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:02:56.580474 19332 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:02:56.583637 19332 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 718b7f31008541a7a2512d539762c544: Sending DeleteTablet for 3 replicas of tablet 4a8bf54df12c45beb69455bac0b5f774
I20260812 07:02:56.584625 19549 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a8bf54df12c45beb69455bac0b5f774 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:56 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:52130
I20260812 07:02:56.584856 20018 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a8bf54df12c45beb69455bac0b5f774 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:56 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:40764
I20260812 07:02:56.585201 19777 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a8bf54df12c45beb69455bac0b5f774 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:56 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:50386
I20260812 07:02:56.586966 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 19442
I20260812 07:02:56.614406 20212 tablet_replica.cc:330] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee: stopping tablet replica
I20260812 07:02:56.614897 20212 raft_consensus.cc:2229] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:02:56.615702 20212 raft_consensus.cc:2258] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:56.620862 20209 tablet_replica.cc:330] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3: stopping tablet replica
I20260812 07:02:56.621256 20212 ts_tablet_manager.cc:1905] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:56.621984 20209 raft_consensus.cc:2229] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3 [term 1 FOLLOWER]: Raft consensus shutting down.
Aug 12 07:02:56 dist-test-slave-lgr5 krb5kdc[19247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.193@KRBTEST.COM
I20260812 07:02:56.625763 20209 raft_consensus.cc:2258] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:56.626626 20210 tablet_replica.cc:330] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db: stopping tablet replica
I20260812 07:02:56.627022 20210 raft_consensus.cc:2229] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:02:56.627494 20210 raft_consensus.cc:2258] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:56.628765 20212 ts_tablet_manager.cc:1918] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:02:56.628907 20212 log.cc:1196] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/4a8bf54df12c45beb69455bac0b5f774
I20260812 07:02:56.629518 20212 ts_tablet_manager.cc:1939] T 4a8bf54df12c45beb69455bac0b5f774 P 1da1499f820b483e850f6f8ab05296ee: Deleting consensus metadata
I20260812 07:02:56.630218 20209 ts_tablet_manager.cc:1905] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:56.631173 19302 catalog_manager.cc:4841] TS 1da1499f820b483e850f6f8ab05296ee (127.11.192.194:41085): tablet 4a8bf54df12c45beb69455bac0b5f774 (table test-table [id=b9d71bbdfa0a44fe93433e1153858a40]) successfully deleted
I20260812 07:02:56.635401 20153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.588149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52144 (local address 127.11.192.193:36355)
0812 07:02:56.588338 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:56.588344 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:56.611603 (+ 23259us) server_negotiation.cc:407] Connection header received
0812 07:02:56.612048 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:56.612055 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:56.612201 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:56.612278 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:56.615931 (+ 3653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.617091 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.619533 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.620205 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.620306 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:56.628934 (+ 8628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:56.628949 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:56.628966 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:56.629018 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:56.632696 (+ 3678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:56.633340 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:56.633346 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:56.633349 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:56.633477 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:56.633828 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:56.633832 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:56.633834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:56.634169 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:56.634455 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:56.634758 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 07:02:56.635031 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:02:56.636153 19655 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:56.637684 20210 ts_tablet_manager.cc:1905] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:56.643321 20210 ts_tablet_manager.cc:1918] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:02:56.643465 20210 log.cc:1196] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/4a8bf54df12c45beb69455bac0b5f774
I20260812 07:02:56.643882 20210 ts_tablet_manager.cc:1939] T 4a8bf54df12c45beb69455bac0b5f774 P d7dced86cb77472bb2606fb6c1fc64db: Deleting consensus metadata
I20260812 07:02:56.645272 19302 catalog_manager.cc:4841] TS d7dced86cb77472bb2606fb6c1fc64db (127.11.192.195:41891): tablet 4a8bf54df12c45beb69455bac0b5f774 (table test-table [id=b9d71bbdfa0a44fe93433e1153858a40]) successfully deleted
I20260812 07:02:56.748648 20209 ts_tablet_manager.cc:1918] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:02:56.748919 20209 log.cc:1196] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518136653111-12035-0/minicluster-data/ts-0/wal/wals/4a8bf54df12c45beb69455bac0b5f774
I20260812 07:02:56.749352 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 19442
I20260812 07:02:56.749533 20209 ts_tablet_manager.cc:1939] T 4a8bf54df12c45beb69455bac0b5f774 P 0bbe2b9734ed439fbbcd823fdfb0b7d3: Deleting consensus metadata
W20260812 07:02:56.773934 19304 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.192.193:36355 (error 108)
I20260812 07:02:56.774550 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 19672
I20260812 07:02:56.776889 20160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.776406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:36355 (local address 127.0.0.1:52158)
0812 07:02:56.776596 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:56.776714 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:36355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260812 07:02:56.777984 19304 catalog_manager.cc:4584] TS 0bbe2b9734ed439fbbcd823fdfb0b7d3 (127.11.192.193:36355): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4a8bf54df12c45beb69455bac0b5f774: Network error: Client connection negotiation failed: client connection to 127.11.192.193:36355: connect: Connection refused (error 111)
Aug 12 07:02:56 dist-test-slave-lgr5 krb5kdc[19247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.194@KRBTEST.COM
I20260812 07:02:56.791460 20157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.775652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50404 (local address 127.11.192.194:41085)
0812 07:02:56.775840 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:56.775846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:56.776064 (+ 218us) server_negotiation.cc:407] Connection header received
0812 07:02:56.776313 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:56.776320 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:56.776406 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:56.776523 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:56.777914 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.779103 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.780525 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.781177 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.781307 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:56.785629 (+ 4322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:56.785641 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:56.785655 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:56.785687 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:56.788748 (+ 3061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:56.789390 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:56.789398 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:56.789400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:56.789540 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:56.789896 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:56.789900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:56.789902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:56.790285 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:56.790586 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:56.790903 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 07:02:56.791138 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:02:56.792181 19882 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:56.831563 20160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.831153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:36355 (local address 127.0.0.1:52174)
0812 07:02:56.831344 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:56.831429 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:36355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 07:02:56.832551 20160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.832205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:36355 (local address 127.0.0.1:52178)
0812 07:02:56.832339 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:56.832406 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:36355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 07:02:56.892963 20160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.892530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:36355 (local address 127.0.0.1:52190)
0812 07:02:56.892728 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:56.892818 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:36355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 07:02:56.894027 20160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.893707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:36355 (local address 127.0.0.1:52204)
0812 07:02:56.893822 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:56.893903 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:36355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 07:02:56.905612 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 19672
I20260812 07:02:56.929158 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 19897
Aug 12 07:02:56 dist-test-slave-lgr5 krb5kdc[19247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.195@KRBTEST.COM
I20260812 07:02:56.943214 20164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.929814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40766 (local address 127.11.192.195:41891)
0812 07:02:56.929989 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:56.929994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:56.930077 (+ 83us) server_negotiation.cc:407] Connection header received
0812 07:02:56.930265 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:56.930269 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:56.930344 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:56.930432 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:56.931698 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.932481 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.933746 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:02:56.934143 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:02:56.934257 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:02:56.938384 (+ 4127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:56.938399 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:56.938417 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:02:56.938452 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:56.941086 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:56.941616 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:56.941623 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:56.941625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:56.941751 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:02:56.942024 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:02:56.942027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:02:56.942029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:02:56.942371 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:56.942590 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:56.942793 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 07:02:56.942949 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:02:56.943795 20111 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:56.974499 20160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.974001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:36355 (local address 127.0.0.1:52218)
0812 07:02:56.974218 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:56.974323 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:36355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 07:02:56.975615 20160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:56.975128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:36355 (local address 127.0.0.1:52232)
0812 07:02:56.975345 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:56.975441 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:36355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260812 07:02:57.031807 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 19897
I20260812 07:02:57.051312 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 19270
I20260812 07:02:57.052481 19384 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:57.159479 20160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:57.158878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:36355 (local address 127.0.0.1:52246)
0812 07:02:57.159087 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:57.159192 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:36355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 07:02:57.159760 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 19270
2026-08-12T07:02:57Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4240 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T07:02:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:02:57Z Disabled control of system clock
I20260812 07:02:57.229947 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:34111
--webserver_interface=127.11.192.254
--webserver_port=0
--builtin_ntp_servers=127.11.192.212:45659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:34111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 07:02:57.546777 20227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:57.547053 20227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:57.547152 20227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:57.554159 20227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:02:57.554282 20227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:57.554366 20227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:57.554435 20227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:02:57.554486 20227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:02:57.563297 20227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:45659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:34111
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:34111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:57.564469 20227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:57.566213 20227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:57.578017 20242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:57.578091 20235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:57.578079 20238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:57.578792 20227 server_base.cc:1034] running on GCE node
I20260812 07:02:57.579476 20227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:57.580793 20227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:57.581992 20227 hybrid_clock.cc:648] HybridClock initialized: now 1786518177581943 us; error 42 us; skew 500 ppm
I20260812 07:02:57.582306 20227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:02:57.584360 20227 webserver.cc:460] Webserver started at http://127.11.192.254:43225/ using document root <none> and password file <none>
I20260812 07:02:57.585044 20227 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:57.585142 20227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:57.585434 20227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:57.588140 20227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "1e3b69920c6f4ad4b06bff815e9cb31c"
format_stamp: "Formatted at 2026-08-12 07:02:57 on dist-test-slave-lgr5"
server_key: "4fcc9f635dc46957cda6b5dc9161487a"
server_key_iv: "be4e5016e29dc38ee912cb8e128abe27"
server_key_version: "encryptionkey@0"
I20260812 07:02:57.588899 20227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "1e3b69920c6f4ad4b06bff815e9cb31c"
format_stamp: "Formatted at 2026-08-12 07:02:57 on dist-test-slave-lgr5"
server_key: "4fcc9f635dc46957cda6b5dc9161487a"
server_key_iv: "be4e5016e29dc38ee912cb8e128abe27"
server_key_version: "encryptionkey@0"
I20260812 07:02:57.593467 20227 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:02:57.596647 20250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:57.597831 20227 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:02:57.597973 20227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "1e3b69920c6f4ad4b06bff815e9cb31c"
format_stamp: "Formatted at 2026-08-12 07:02:57 on dist-test-slave-lgr5"
server_key: "4fcc9f635dc46957cda6b5dc9161487a"
server_key_iv: "be4e5016e29dc38ee912cb8e128abe27"
server_key_version: "encryptionkey@0"
I20260812 07:02:57.598158 20227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:57.603480 20227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:57.604212 20227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:57.604476 20227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:57.620950 20227 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:34111
I20260812 07:02:57.621002 20332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:34111 every 8 connection(s)
I20260812 07:02:57.622123 20227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:02:57.625643 20333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:57.625691 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 20227
I20260812 07:02:57.625902 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:02:57.626683 12035 external_mini_cluster.cc:1428] Setting key 65e6b54977ee437de78c9ff6bb4b6250
I20260812 07:02:57.634605 20339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:57.628688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37504 (local address 127.11.192.254:34111)
0812 07:02:57.629610 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:57.629620 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:57.629660 (+ 40us) server_negotiation.cc:407] Connection header received
0812 07:02:57.631455 (+ 1795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:57.631476 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:57.631853 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:57.632209 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:57.633202 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:57.633208 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:57.633219 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:02:57.633233 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:57.633292 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:02:57.633306 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:57.633460 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:57.633572 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 07:02:57.633781 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":350,"threads_started":1}
I20260812 07:02:57.638226 20333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c: Bootstrap starting.
I20260812 07:02:57.641026 20333 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:57.641953 20333 log.cc:824] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:57.644353 20333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c: No bootstrap required, opened a new log
I20260812 07:02:57.651080 20333 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3b69920c6f4ad4b06bff815e9cb31c" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 34111 } }
I20260812 07:02:57.651389 20333 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:57.651490 20333 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e3b69920c6f4ad4b06bff815e9cb31c, State: Initialized, Role: FOLLOWER
I20260812 07:02:57.652055 20333 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3b69920c6f4ad4b06bff815e9cb31c" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 34111 } }
I20260812 07:02:57.652339 20333 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:02:57.652486 20333 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:02:57.652630 20333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:57.654579 20333 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3b69920c6f4ad4b06bff815e9cb31c" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 34111 } }
I20260812 07:02:57.655019 20333 leader_election.cc:302] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e3b69920c6f4ad4b06bff815e9cb31c; no voters:
I20260812 07:02:57.655561 20333 leader_election.cc:288] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:02:57.655687 20342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:02:57.656324 20342 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c [term 1 LEADER]: Becoming Leader. State: Replica: 1e3b69920c6f4ad4b06bff815e9cb31c, State: Running, Role: LEADER
I20260812 07:02:57.656873 20342 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3b69920c6f4ad4b06bff815e9cb31c" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 34111 } }
I20260812 07:02:57.657516 20333 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:02:57.659229 20344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e3b69920c6f4ad4b06bff815e9cb31c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3b69920c6f4ad4b06bff815e9cb31c" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 34111 } } }
I20260812 07:02:57.659608 20344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:57.659401 20347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e3b69920c6f4ad4b06bff815e9cb31c. Latest consensus state: current_term: 1 leader_uuid: "1e3b69920c6f4ad4b06bff815e9cb31c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3b69920c6f4ad4b06bff815e9cb31c" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 34111 } } }
I20260812 07:02:57.659749 20347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c [sys.catalog]: This master's current role is: LEADER
I20260812 07:02:57.666947 20353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:02:57.671770 20353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:02:57.679651 20353 catalog_manager.cc:1347] Generated new cluster ID: 3e63fff8180840029a3b24e38e5a3835
I20260812 07:02:57.679747 20353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:02:57.690932 20353 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:02:57.691274 20353 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:02:57.697687 20353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c: Generated new TSK 0
I20260812 07:02:57.698335 20353 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:02:57.703099 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:34111
--builtin_ntp_servers=127.11.192.212:45659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 07:02:58.027256 20376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:58.027498 20376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:58.027595 20376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:58.034507 20376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:58.034591 20376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:58.034703 20376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:02:58.043351 20376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:45659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:34111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:58.044430 20376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:58.046084 20376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:58.058254 20390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:58.058257 20388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:58.058547 20387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:58.059168 20376 server_base.cc:1034] running on GCE node
I20260812 07:02:58.059690 20376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:58.060549 20376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:58.061734 20376 hybrid_clock.cc:648] HybridClock initialized: now 1786518178061695 us; error 33 us; skew 500 ppm
I20260812 07:02:58.062013 20376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:02:58.063977 20376 webserver.cc:460] Webserver started at http://127.11.192.193:40797/ using document root <none> and password file <none>
I20260812 07:02:58.064634 20376 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:58.064724 20376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:58.065034 20376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:58.067689 20376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "a50e33df44ff474d9a0d4562b920e4f1"
format_stamp: "Formatted at 2026-08-12 07:02:58 on dist-test-slave-lgr5"
server_key: "ed422d4e8830988353cfc295a08b3b11"
server_key_iv: "e1b4721e268bc0bd91eaef624a127ec3"
server_key_version: "encryptionkey@0"
I20260812 07:02:58.068475 20376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "a50e33df44ff474d9a0d4562b920e4f1"
format_stamp: "Formatted at 2026-08-12 07:02:58 on dist-test-slave-lgr5"
server_key: "ed422d4e8830988353cfc295a08b3b11"
server_key_iv: "e1b4721e268bc0bd91eaef624a127ec3"
server_key_version: "encryptionkey@0"
I20260812 07:02:58.073086 20376 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:02:58.076244 20400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:58.077405 20376 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 07:02:58.077548 20376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "a50e33df44ff474d9a0d4562b920e4f1"
format_stamp: "Formatted at 2026-08-12 07:02:58 on dist-test-slave-lgr5"
server_key: "ed422d4e8830988353cfc295a08b3b11"
server_key_iv: "e1b4721e268bc0bd91eaef624a127ec3"
server_key_version: "encryptionkey@0"
I20260812 07:02:58.077742 20376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:58.083037 20376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:58.083736 20376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:58.083961 20376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:58.084843 20376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:02:58.086110 20376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:58.086199 20376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:58.086300 20376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:58.086371 20376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:58.115439 20376 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:45323
I20260812 07:02:58.115495 20580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:45323 every 8 connection(s)
I20260812 07:02:58.116655 20376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:02:58.117162 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 20376
I20260812 07:02:58.117369 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:02:58.118130 12035 external_mini_cluster.cc:1428] Setting key c7680764a21ab2a979e5e8bf8aa1113b
I20260812 07:02:58.121300 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:34111
--builtin_ntp_servers=127.11.192.212:45659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 07:02:58.125243 20339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:58.119266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:38429 (local address 127.11.192.254:34111)
0812 07:02:58.119419 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:58.119423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:58.120943 (+ 1520us) server_negotiation.cc:407] Connection header received
0812 07:02:58.123134 (+ 2191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:58.123138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:58.123189 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:58.123268 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:58.124643 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:58.124647 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:58.124650 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:02:58.124665 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:58.124696 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:02:58.124708 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:58.124796 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:58.125001 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 07:02:58.125049 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:02:58.126308 20585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:58.119674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:34111 (local address 127.11.192.193:38429)
0812 07:02:58.120778 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:58.120833 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:58.122866 (+ 2033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:58.123515 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:58.123525 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:58.123631 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:58.124326 (+ 695us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:02:58.124330 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:58.124373 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:02:58.124483 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:58.124793 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:58.124802 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:58.124808 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 07:02:58.125072 (+ 264us) client_negotiation.cc:240] Negotiation successful
0812 07:02:58.125293 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":345,"threads_started":1}
I20260812 07:02:58.127624 20581 heartbeater.cc:347] Connected to a master server at 127.11.192.254:34111
I20260812 07:02:58.127871 20581 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:58.128427 20581 heartbeater.cc:511] Master 127.11.192.254:34111 requested a full tablet report, sending...
I20260812 07:02:58.130352 20277 ts_manager.cc:194] Registered new tserver with Master: a50e33df44ff474d9a0d4562b920e4f1 (127.11.192.193:45323)
W20260812 07:02:58.438335 20586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:58.438597 20586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:58.438712 20586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:58.445626 20586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:58.445724 20586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:58.445868 20586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:02:58.454425 20586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:45659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:34111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:58.455538 20586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:58.457199 20586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:58.469291 20597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:58.469412 20598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:58.469446 20605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:58.470139 20586 server_base.cc:1034] running on GCE node
I20260812 07:02:58.470674 20586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:58.471465 20586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:58.472651 20586 hybrid_clock.cc:648] HybridClock initialized: now 1786518178472611 us; error 31 us; skew 500 ppm
I20260812 07:02:58.472940 20586 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:02:58.475018 20586 webserver.cc:460] Webserver started at http://127.11.192.194:37727/ using document root <none> and password file <none>
I20260812 07:02:58.475646 20586 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:58.475736 20586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:58.476042 20586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:58.478662 20586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "f2605dee56bb4b989c94602205a2e29e"
format_stamp: "Formatted at 2026-08-12 07:02:58 on dist-test-slave-lgr5"
server_key: "e94dd54f7ef8d633ec7bc7adb69e2a01"
server_key_iv: "7c651aba6b392eb90f73079767422541"
server_key_version: "encryptionkey@0"
I20260812 07:02:58.479398 20586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "f2605dee56bb4b989c94602205a2e29e"
format_stamp: "Formatted at 2026-08-12 07:02:58 on dist-test-slave-lgr5"
server_key: "e94dd54f7ef8d633ec7bc7adb69e2a01"
server_key_iv: "7c651aba6b392eb90f73079767422541"
server_key_version: "encryptionkey@0"
I20260812 07:02:58.484247 20586 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260812 07:02:58.487344 20615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:58.488591 20586 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:02:58.488737 20586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "f2605dee56bb4b989c94602205a2e29e"
format_stamp: "Formatted at 2026-08-12 07:02:58 on dist-test-slave-lgr5"
server_key: "e94dd54f7ef8d633ec7bc7adb69e2a01"
server_key_iv: "7c651aba6b392eb90f73079767422541"
server_key_version: "encryptionkey@0"
I20260812 07:02:58.488924 20586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:58.494455 20586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:58.495229 20586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:58.495498 20586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:58.496457 20586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:02:58.497833 20586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:58.497938 20586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:58.498063 20586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:58.498142 20586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:58.529660 20586 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:46303
I20260812 07:02:58.529704 20803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:46303 every 8 connection(s)
I20260812 07:02:58.530901 20586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:02:58.535264 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 20586
I20260812 07:02:58.535446 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:02:58.535899 12035 external_mini_cluster.cc:1428] Setting key c367ff6554d2fc19c651ed879cb4002b
I20260812 07:02:58.538784 20339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:58.533282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:48913 (local address 127.11.192.254:34111)
0812 07:02:58.533499 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:58.533503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:58.534769 (+ 1266us) server_negotiation.cc:407] Connection header received
0812 07:02:58.536490 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:58.536494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:58.536554 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:58.536654 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:58.538007 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:58.538013 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:58.538018 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:02:58.538041 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:58.538086 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:02:58.538105 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:58.538243 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:58.538397 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 07:02:58.538465 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:02:58.539125 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:34111
--builtin_ntp_servers=127.11.192.212:45659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 07:02:58.539508 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:58.533764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:34111 (local address 127.11.192.194:48913)
0812 07:02:58.534621 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:58.534654 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:58.536223 (+ 1569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:58.536868 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:58.536876 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:58.536972 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:58.537658 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:02:58.537660 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:58.537705 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:02:58.537812 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:58.538212 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:58.538220 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:58.538225 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 07:02:58.538429 (+ 204us) client_negotiation.cc:240] Negotiation successful
0812 07:02:58.538611 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":262,"threads_started":1}
I20260812 07:02:58.540773 20804 heartbeater.cc:347] Connected to a master server at 127.11.192.254:34111
I20260812 07:02:58.541092 20804 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:58.541775 20804 heartbeater.cc:511] Master 127.11.192.254:34111 requested a full tablet report, sending...
I20260812 07:02:58.543206 20277 ts_manager.cc:194] Registered new tserver with Master: f2605dee56bb4b989c94602205a2e29e (127.11.192.194:46303)
W20260812 07:02:58.862428 20809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:02:58.862699 20809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:02:58.862874 20809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:02:58.870874 20809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:02:58.870971 20809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:02:58.871091 20809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:02:58.879812 20809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:45659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:34111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:02:58.880980 20809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:02:58.882843 20809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:02:58.895488 20817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:58.895555 20816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:02:58.895404 20821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:02:58.896450 20809 server_base.cc:1034] running on GCE node
I20260812 07:02:58.897022 20809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:02:58.897845 20809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:02:58.899039 20809 hybrid_clock.cc:648] HybridClock initialized: now 1786518178898994 us; error 35 us; skew 500 ppm
I20260812 07:02:58.899322 20809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:02:58.901407 20809 webserver.cc:460] Webserver started at http://127.11.192.195:42725/ using document root <none> and password file <none>
I20260812 07:02:58.902107 20809 fs_manager.cc:359] Metadata directory not provided
I20260812 07:02:58.902204 20809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:02:58.902513 20809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:02:58.905203 20809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "f395f23ebeff4dc2b3ddb55f871b75f3"
format_stamp: "Formatted at 2026-08-12 07:02:58 on dist-test-slave-lgr5"
server_key: "59ad66ed81aac097e645d2b25c74610b"
server_key_iv: "39bba0025e26c4e2408d5ab8d0c51ff4"
server_key_version: "encryptionkey@0"
I20260812 07:02:58.905936 20809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "f395f23ebeff4dc2b3ddb55f871b75f3"
format_stamp: "Formatted at 2026-08-12 07:02:58 on dist-test-slave-lgr5"
server_key: "59ad66ed81aac097e645d2b25c74610b"
server_key_iv: "39bba0025e26c4e2408d5ab8d0c51ff4"
server_key_version: "encryptionkey@0"
I20260812 07:02:58.910705 20809 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 07:02:58.913950 20832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:58.915186 20809 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:02:58.915337 20809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "f395f23ebeff4dc2b3ddb55f871b75f3"
format_stamp: "Formatted at 2026-08-12 07:02:58 on dist-test-slave-lgr5"
server_key: "59ad66ed81aac097e645d2b25c74610b"
server_key_iv: "39bba0025e26c4e2408d5ab8d0c51ff4"
server_key_version: "encryptionkey@0"
I20260812 07:02:58.915561 20809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:02:58.920879 20809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:02:58.921711 20809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:02:58.921977 20809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:02:58.922799 20809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:02:58.924068 20809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:02:58.924191 20809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:58.924292 20809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:02:58.924363 20809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:02:58.953563 20809 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:42115
I20260812 07:02:58.953619 21027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:42115 every 8 connection(s)
I20260812 07:02:58.954739 20809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:02:58.959373 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 20809
I20260812 07:02:58.959548 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:02:58.960068 12035 external_mini_cluster.cc:1428] Setting key 73874cc7ab80eabdcc6ff898765e4b21
I20260812 07:02:58.962936 20339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:58.957108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:33963 (local address 127.11.192.254:34111)
0812 07:02:58.957284 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:58.957289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:58.958596 (+ 1307us) server_negotiation.cc:407] Connection header received
0812 07:02:58.960733 (+ 2137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:58.960740 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:58.960811 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:58.960908 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:58.962268 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:58.962275 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:58.962280 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:02:58.962301 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:58.962342 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:02:58.962357 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:58.962488 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:58.962612 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 07:02:58.962675 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:02:58.963721 21034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:58.957607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:34111 (local address 127.11.192.195:33963)
0812 07:02:58.958457 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:58.958506 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:58.960426 (+ 1920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:58.961180 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:58.961187 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:58.961275 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:58.961918 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:02:58.961920 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:58.961964 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:02:58.962082 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:58.962441 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:58.962448 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:58.962453 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 07:02:58.962654 (+ 201us) client_negotiation.cc:240] Negotiation successful
0812 07:02:58.962826 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":260,"threads_started":1}
I20260812 07:02:58.964831 21028 heartbeater.cc:347] Connected to a master server at 127.11.192.254:34111
I20260812 07:02:58.965076 21028 heartbeater.cc:464] Registering TS with master...
I20260812 07:02:58.965579 21028 heartbeater.cc:511] Master 127.11.192.254:34111 requested a full tablet report, sending...
I20260812 07:02:58.966740 20277 ts_manager.cc:194] Registered new tserver with Master: f395f23ebeff4dc2b3ddb55f871b75f3 (127.11.192.195:42115)
I20260812 07:02:58.975131 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:02:58.981436 20339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:58.979242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37518 (local address 127.11.192.254:34111)
0812 07:02:58.979379 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:58.979382 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:58.979920 (+ 538us) server_negotiation.cc:407] Connection header received
0812 07:02:58.980051 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:58.980055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:58.980134 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:58.980204 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:58.980999 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:58.981003 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:58.981006 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:02:58.981021 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:58.981049 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:02:58.981060 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:58.981150 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:58.981199 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 07:02:58.981242 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 07:02:58.984778 20277 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:02:58.991365 20277 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:02:59.008373 21063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:59.002193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:46303 (local address 127.0.0.1:55738)
0812 07:02:59.004144 (+ 1951us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:59.004180 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:59.004391 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:59.006054 (+ 1663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:59.006063 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:59.006154 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:59.006893 (+ 739us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:02:59.006897 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:59.006928 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:02:59.007024 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:59.007589 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:59.007601 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:59.007612 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 07:02:59.007784 (+ 172us) client_negotiation.cc:240] Negotiation successful
0812 07:02:59.007875 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1718,"thread_start_us":2054,"threads_started":1}
I20260812 07:02:59.009032 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:59.002324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55738 (local address 127.11.192.194:46303)
0812 07:02:59.003115 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:59.003133 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:59.004278 (+ 1145us) server_negotiation.cc:407] Connection header received
0812 07:02:59.004545 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:59.004555 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:59.004779 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:59.004929 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:59.007238 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:59.007249 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:59.007264 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:02:59.007291 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:59.007350 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:02:59.007371 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:59.007534 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:59.008601 (+ 1067us) server_negotiation.cc:299] Negotiation successful
0812 07:02:59.008701 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":787,"threads_started":1}
I20260812 07:02:59.009128 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:59.002911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47132 (local address 127.11.192.193:45323)
0812 07:02:59.003614 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:59.003624 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:59.004792 (+ 1168us) server_negotiation.cc:407] Connection header received
0812 07:02:59.004876 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:59.004884 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:59.005097 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:59.006291 (+ 1194us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:59.007961 (+ 1670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:59.007969 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:59.007982 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:02:59.008013 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:59.008070 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:02:59.008123 (+ 53us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:59.008253 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:59.008736 (+ 483us) server_negotiation.cc:299] Negotiation successful
0812 07:02:59.008835 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":418,"threads_started":1}
I20260812 07:02:59.009825 21058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:59.002563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:45323 (local address 127.0.0.1:47132)
0812 07:02:59.004553 (+ 1990us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:59.004568 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:59.004663 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:59.007018 (+ 2355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:59.007023 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:59.007046 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:59.007692 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:02:59.007695 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:59.007711 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:02:59.007754 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:59.008318 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:59.008326 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:59.008331 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 07:02:59.009401 (+ 1070us) client_negotiation.cc:240] Negotiation successful
0812 07:02:59.009453 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1846,"thread_start_us":1209,"threads_started":1}
I20260812 07:02:59.010666 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:59.004617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:42115 (local address 127.0.0.1:40552)
0812 07:02:59.007627 (+ 3010us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:59.007643 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:59.007742 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:59.008191 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:59.008196 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:59.008222 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:59.009138 (+ 916us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:02:59.009141 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:59.009156 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:02:59.009200 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:59.010107 (+ 907us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:59.010111 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:59.010116 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 07:02:59.010242 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 07:02:59.010290 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2885,"thread_start_us":2176,"threads_started":1}
I20260812 07:02:59.011301 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:59.004904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40552 (local address 127.11.192.195:42115)
0812 07:02:59.007579 (+ 2675us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:59.007588 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:59.007737 (+ 149us) server_negotiation.cc:407] Connection header received
0812 07:02:59.007865 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:59.007873 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:59.008038 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:59.008183 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:59.009819 (+ 1636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:59.009825 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:59.009833 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:02:59.009845 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:59.009877 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:02:59.009889 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:59.010048 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:59.010966 (+ 918us) server_negotiation.cc:299] Negotiation successful
0812 07:02:59.011045 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2501,"thread_start_us":2333,"threads_started":1}
I20260812 07:02:59.012463 20690 tablet_service.cc:1467] Processing CreateTablet for tablet b074b9c44ffc4a1cb6e2851cf2697de0 (DEFAULT_TABLE table=test-table [id=396133dc3bc048729b2634cd8dc8f90f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:59.014017 20690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b074b9c44ffc4a1cb6e2851cf2697de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:59.013897 20478 tablet_service.cc:1467] Processing CreateTablet for tablet b074b9c44ffc4a1cb6e2851cf2697de0 (DEFAULT_TABLE table=test-table [id=396133dc3bc048729b2634cd8dc8f90f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:59.014899 20478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b074b9c44ffc4a1cb6e2851cf2697de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:59.014729 20916 tablet_service.cc:1467] Processing CreateTablet for tablet b074b9c44ffc4a1cb6e2851cf2697de0 (DEFAULT_TABLE table=test-table [id=396133dc3bc048729b2634cd8dc8f90f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:02:59.016074 20916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b074b9c44ffc4a1cb6e2851cf2697de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:02:59.025707 21071 tablet_bootstrap.cc:492] T b074b9c44ffc4a1cb6e2851cf2697de0 P a50e33df44ff474d9a0d4562b920e4f1: Bootstrap starting.
I20260812 07:02:59.028043 21072 tablet_bootstrap.cc:492] T b074b9c44ffc4a1cb6e2851cf2697de0 P f2605dee56bb4b989c94602205a2e29e: Bootstrap starting.
I20260812 07:02:59.029053 21071 tablet_bootstrap.cc:654] T b074b9c44ffc4a1cb6e2851cf2697de0 P a50e33df44ff474d9a0d4562b920e4f1: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:59.029539 21073 tablet_bootstrap.cc:492] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3: Bootstrap starting.
I20260812 07:02:59.030319 21071 log.cc:824] T b074b9c44ffc4a1cb6e2851cf2697de0 P a50e33df44ff474d9a0d4562b920e4f1: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:59.031457 21072 tablet_bootstrap.cc:654] T b074b9c44ffc4a1cb6e2851cf2697de0 P f2605dee56bb4b989c94602205a2e29e: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:59.032449 21073 tablet_bootstrap.cc:654] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3: Neither blocks nor log segments found. Creating new log.
I20260812 07:02:59.032783 21072 log.cc:824] T b074b9c44ffc4a1cb6e2851cf2697de0 P f2605dee56bb4b989c94602205a2e29e: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:59.033455 21073 log.cc:824] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3: Log is configured to *not* fsync() on all Append() calls
I20260812 07:02:59.035586 21072 tablet_bootstrap.cc:492] T b074b9c44ffc4a1cb6e2851cf2697de0 P f2605dee56bb4b989c94602205a2e29e: No bootstrap required, opened a new log
I20260812 07:02:59.035830 21072 ts_tablet_manager.cc:1397] T b074b9c44ffc4a1cb6e2851cf2697de0 P f2605dee56bb4b989c94602205a2e29e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 07:02:59.036453 21071 tablet_bootstrap.cc:492] T b074b9c44ffc4a1cb6e2851cf2697de0 P a50e33df44ff474d9a0d4562b920e4f1: No bootstrap required, opened a new log
I20260812 07:02:59.036453 21073 tablet_bootstrap.cc:492] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3: No bootstrap required, opened a new log
I20260812 07:02:59.036715 21071 ts_tablet_manager.cc:1397] T b074b9c44ffc4a1cb6e2851cf2697de0 P a50e33df44ff474d9a0d4562b920e4f1: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260812 07:02:59.036720 21073 ts_tablet_manager.cc:1397] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 07:02:59.042832 21072 raft_consensus.cc:348] T b074b9c44ffc4a1cb6e2851cf2697de0 P f2605dee56bb4b989c94602205a2e29e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2605dee56bb4b989c94602205a2e29e" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 46303 } } peers { permanent_uuid: "f395f23ebeff4dc2b3ddb55f871b75f3" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42115 } } peers { permanent_uuid: "a50e33df44ff474d9a0d4562b920e4f1" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45323 } }
I20260812 07:02:59.043162 21072 raft_consensus.cc:374] T b074b9c44ffc4a1cb6e2851cf2697de0 P f2605dee56bb4b989c94602205a2e29e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:59.043290 21072 raft_consensus.cc:729] T b074b9c44ffc4a1cb6e2851cf2697de0 P f2605dee56bb4b989c94602205a2e29e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2605dee56bb4b989c94602205a2e29e, State: Initialized, Role: FOLLOWER
I20260812 07:02:59.043821 21072 consensus_queue.cc:260] T b074b9c44ffc4a1cb6e2851cf2697de0 P f2605dee56bb4b989c94602205a2e29e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2605dee56bb4b989c94602205a2e29e" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 46303 } } peers { permanent_uuid: "f395f23ebeff4dc2b3ddb55f871b75f3" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42115 } } peers { permanent_uuid: "a50e33df44ff474d9a0d4562b920e4f1" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45323 } }
I20260812 07:02:59.044770 20804 heartbeater.cc:503] Master 127.11.192.254:34111 was elected leader, sending a full tablet report...
I20260812 07:02:59.045408 21072 ts_tablet_manager.cc:1428] T b074b9c44ffc4a1cb6e2851cf2697de0 P f2605dee56bb4b989c94602205a2e29e: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260812 07:02:59.046764 21073 raft_consensus.cc:348] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2605dee56bb4b989c94602205a2e29e" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 46303 } } peers { permanent_uuid: "f395f23ebeff4dc2b3ddb55f871b75f3" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42115 } } peers { permanent_uuid: "a50e33df44ff474d9a0d4562b920e4f1" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45323 } }
I20260812 07:02:59.046697 21071 raft_consensus.cc:348] T b074b9c44ffc4a1cb6e2851cf2697de0 P a50e33df44ff474d9a0d4562b920e4f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2605dee56bb4b989c94602205a2e29e" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 46303 } } peers { permanent_uuid: "f395f23ebeff4dc2b3ddb55f871b75f3" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42115 } } peers { permanent_uuid: "a50e33df44ff474d9a0d4562b920e4f1" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45323 } }
I20260812 07:02:59.047137 21073 raft_consensus.cc:374] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:59.047137 21071 raft_consensus.cc:374] T b074b9c44ffc4a1cb6e2851cf2697de0 P a50e33df44ff474d9a0d4562b920e4f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:02:59.047237 21073 raft_consensus.cc:729] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f395f23ebeff4dc2b3ddb55f871b75f3, State: Initialized, Role: FOLLOWER
I20260812 07:02:59.047236 21071 raft_consensus.cc:729] T b074b9c44ffc4a1cb6e2851cf2697de0 P a50e33df44ff474d9a0d4562b920e4f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a50e33df44ff474d9a0d4562b920e4f1, State: Initialized, Role: FOLLOWER
I20260812 07:02:59.047802 21073 consensus_queue.cc:260] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2605dee56bb4b989c94602205a2e29e" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 46303 } } peers { permanent_uuid: "f395f23ebeff4dc2b3ddb55f871b75f3" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42115 } } peers { permanent_uuid: "a50e33df44ff474d9a0d4562b920e4f1" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45323 } }
I20260812 07:02:59.047802 21071 consensus_queue.cc:260] T b074b9c44ffc4a1cb6e2851cf2697de0 P a50e33df44ff474d9a0d4562b920e4f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2605dee56bb4b989c94602205a2e29e" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 46303 } } peers { permanent_uuid: "f395f23ebeff4dc2b3ddb55f871b75f3" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42115 } } peers { permanent_uuid: "a50e33df44ff474d9a0d4562b920e4f1" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45323 } }
I20260812 07:02:59.048946 20581 heartbeater.cc:503] Master 127.11.192.254:34111 was elected leader, sending a full tablet report...
I20260812 07:02:59.049398 21028 heartbeater.cc:503] Master 127.11.192.254:34111 was elected leader, sending a full tablet report...
I20260812 07:02:59.049561 21071 ts_tablet_manager.cc:1428] T b074b9c44ffc4a1cb6e2851cf2697de0 P a50e33df44ff474d9a0d4562b920e4f1: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260812 07:02:59.051038 21073 ts_tablet_manager.cc:1428] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
W20260812 07:02:59.118736 20584 tablet.cc:2369] T b074b9c44ffc4a1cb6e2851cf2697de0 P a50e33df44ff474d9a0d4562b920e4f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:02:59.204353 21082 raft_consensus.cc:482] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:02:59.204639 21082 raft_consensus.cc:504] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2605dee56bb4b989c94602205a2e29e" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 46303 } } peers { permanent_uuid: "f395f23ebeff4dc2b3ddb55f871b75f3" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42115 } } peers { permanent_uuid: "a50e33df44ff474d9a0d4562b920e4f1" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45323 } }
W20260812 07:02:59.206272 21029 tablet.cc:2369] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:02:59.206437 21082 leader_election.cc:288] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2605dee56bb4b989c94602205a2e29e (127.11.192.194:46303), a50e33df44ff474d9a0d4562b920e4f1 (127.11.192.193:45323)
I20260812 07:02:59.209159 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:59.207263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:33137 (local address 127.11.192.194:46303)
0812 07:02:59.207409 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:59.207413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:59.207428 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:02:59.207480 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:59.207484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:59.207528 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:59.207604 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:59.208571 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:59.208576 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:59.208580 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:02:59.208605 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:59.208652 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:02:59.208668 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:59.208791 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:59.208886 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 07:02:59.208949 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:02:59.209168 21034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:59.206604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:46303 (local address 127.11.192.195:33137)
0812 07:02:59.206819 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:59.206840 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:59.206943 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:59.207673 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:59.207678 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:59.207703 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:59.208348 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:02:59.208350 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:59.208366 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:02:59.208410 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:59.208780 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:59.208784 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:59.208788 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 07:02:59.208867 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 07:02:59.208920 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 07:02:59.209653 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:59.207128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:57855 (local address 127.11.192.193:45323)
0812 07:02:59.207306 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:59.207312 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:59.207767 (+ 455us) server_negotiation.cc:407] Connection header received
0812 07:02:59.207908 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:59.207912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:59.207959 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:59.208040 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:59.208974 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:59.208980 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:59.208985 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:02:59.209003 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:59.209040 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:02:59.209056 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:59.209163 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:59.209324 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 07:02:59.209390 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:02:59.210182 21084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:59.206757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:45323 (local address 127.11.192.195:57855)
0812 07:02:59.207665 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:59.207682 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:02:59.207790 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:02:59.208123 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:02:59.208128 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:02:59.208151 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:02:59.208752 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:02:59.208755 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:02:59.208770 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:02:59.208820 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:02:59.209158 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:02:59.209163 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:02:59.209168 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 07:02:59.209873 (+ 705us) client_negotiation.cc:240] Negotiation successful
0812 07:02:59.209912 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"thread_start_us":597,"threads_started":1}
I20260812 07:02:59.210124 20727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b074b9c44ffc4a1cb6e2851cf2697de0" candidate_uuid: "f395f23ebeff4dc2b3ddb55f871b75f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2605dee56bb4b989c94602205a2e29e" is_pre_election: true
I20260812 07:02:59.210577 20727 raft_consensus.cc:2454] T b074b9c44ffc4a1cb6e2851cf2697de0 P f2605dee56bb4b989c94602205a2e29e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f395f23ebeff4dc2b3ddb55f871b75f3 in term 0.
I20260812 07:02:59.211112 20836 leader_election.cc:302] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f2605dee56bb4b989c94602205a2e29e, f395f23ebeff4dc2b3ddb55f871b75f3; no voters:
I20260812 07:02:59.211097 20507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b074b9c44ffc4a1cb6e2851cf2697de0" candidate_uuid: "f395f23ebeff4dc2b3ddb55f871b75f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a50e33df44ff474d9a0d4562b920e4f1" is_pre_election: true
I20260812 07:02:59.211448 21082 raft_consensus.cc:2781] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:02:59.211609 21082 raft_consensus.cc:482] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:02:59.211599 20507 raft_consensus.cc:2454] T b074b9c44ffc4a1cb6e2851cf2697de0 P a50e33df44ff474d9a0d4562b920e4f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f395f23ebeff4dc2b3ddb55f871b75f3 in term 0.
I20260812 07:02:59.211715 21082 raft_consensus.cc:3037] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:59.214080 21082 raft_consensus.cc:504] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2605dee56bb4b989c94602205a2e29e" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 46303 } } peers { permanent_uuid: "f395f23ebeff4dc2b3ddb55f871b75f3" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42115 } } peers { permanent_uuid: "a50e33df44ff474d9a0d4562b920e4f1" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45323 } }
I20260812 07:02:59.214576 21082 leader_election.cc:288] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3 [CANDIDATE]: Term 1 election: Requested vote from peers f2605dee56bb4b989c94602205a2e29e (127.11.192.194:46303), a50e33df44ff474d9a0d4562b920e4f1 (127.11.192.193:45323)
I20260812 07:02:59.214984 20727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b074b9c44ffc4a1cb6e2851cf2697de0" candidate_uuid: "f395f23ebeff4dc2b3ddb55f871b75f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2605dee56bb4b989c94602205a2e29e"
I20260812 07:02:59.215037 20507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b074b9c44ffc4a1cb6e2851cf2697de0" candidate_uuid: "f395f23ebeff4dc2b3ddb55f871b75f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a50e33df44ff474d9a0d4562b920e4f1"
I20260812 07:02:59.215162 20727 raft_consensus.cc:3037] T b074b9c44ffc4a1cb6e2851cf2697de0 P f2605dee56bb4b989c94602205a2e29e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:59.215221 20507 raft_consensus.cc:3037] T b074b9c44ffc4a1cb6e2851cf2697de0 P a50e33df44ff474d9a0d4562b920e4f1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:02:59.217324 20727 raft_consensus.cc:2454] T b074b9c44ffc4a1cb6e2851cf2697de0 P f2605dee56bb4b989c94602205a2e29e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f395f23ebeff4dc2b3ddb55f871b75f3 in term 1.
I20260812 07:02:59.217428 20507 raft_consensus.cc:2454] T b074b9c44ffc4a1cb6e2851cf2697de0 P a50e33df44ff474d9a0d4562b920e4f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f395f23ebeff4dc2b3ddb55f871b75f3 in term 1.
I20260812 07:02:59.217682 20836 leader_election.cc:302] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f2605dee56bb4b989c94602205a2e29e, f395f23ebeff4dc2b3ddb55f871b75f3; no voters:
I20260812 07:02:59.217952 21082 raft_consensus.cc:2781] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:02:59.218339 21082 raft_consensus.cc:686] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3 [term 1 LEADER]: Becoming Leader. State: Replica: f395f23ebeff4dc2b3ddb55f871b75f3, State: Running, Role: LEADER
I20260812 07:02:59.218735 21082 consensus_queue.cc:237] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2605dee56bb4b989c94602205a2e29e" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 46303 } } peers { permanent_uuid: "f395f23ebeff4dc2b3ddb55f871b75f3" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42115 } } peers { permanent_uuid: "a50e33df44ff474d9a0d4562b920e4f1" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45323 } }
I20260812 07:02:59.223282 20277 catalog_manager.cc:5495] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to f395f23ebeff4dc2b3ddb55f871b75f3 (127.11.192.195). New cstate: current_term: 1 leader_uuid: "f395f23ebeff4dc2b3ddb55f871b75f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2605dee56bb4b989c94602205a2e29e" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 46303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f395f23ebeff4dc2b3ddb55f871b75f3" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a50e33df44ff474d9a0d4562b920e4f1" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45323 } health_report { overall_health: UNKNOWN } } }
I20260812 07:02:59.241537 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:59.239869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40568 (local address 127.11.192.195:42115)
0812 07:02:59.240009 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:59.240013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:59.240051 (+ 38us) server_negotiation.cc:407] Connection header received
0812 07:02:59.240266 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:59.240270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:59.240339 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:59.240417 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:59.241146 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:59.241150 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:59.241153 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:02:59.241164 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:59.241186 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:02:59.241196 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:59.241283 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:59.241334 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 07:02:59.241374 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 07:02:59.248714 20507 raft_consensus.cc:1261] T b074b9c44ffc4a1cb6e2851cf2697de0 P a50e33df44ff474d9a0d4562b920e4f1 [term 1 FOLLOWER]: Refusing update from remote peer f395f23ebeff4dc2b3ddb55f871b75f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:59.248720 20727 raft_consensus.cc:1261] T b074b9c44ffc4a1cb6e2851cf2697de0 P f2605dee56bb4b989c94602205a2e29e [term 1 FOLLOWER]: Refusing update from remote peer f395f23ebeff4dc2b3ddb55f871b75f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:02:59.249529 21085 consensus_queue.cc:1035] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2605dee56bb4b989c94602205a2e29e" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 46303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:02:59.249831 21082 consensus_queue.cc:1035] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a50e33df44ff474d9a0d4562b920e4f1" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:02:59.260377 21091 mvcc.cc:205] Tried to move back new op lower bound from 7317578462194356224 to 7317578462083538944. Current Snapshot: MvccSnapshot[applied={T|T < 7317578462194356224}]
I20260812 07:02:59.263787 21095 mvcc.cc:205] Tried to move back new op lower bound from 7317578462194356224 to 7317578462083538944. Current Snapshot: MvccSnapshot[applied={T|T < 7317578462194356224 or (T in {7317578462194356224})}]
I20260812 07:02:59.272245 20275 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:02:59.272572 20275 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:02:59.276206 20275 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e3b69920c6f4ad4b06bff815e9cb31c: Sending DeleteTablet for 3 replicas of tablet b074b9c44ffc4a1cb6e2851cf2697de0
I20260812 07:02:59.277387 20478 tablet_service.cc:1514] Processing DeleteTablet for tablet b074b9c44ffc4a1cb6e2851cf2697de0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:59 UTC) from {username='slave'} at 127.0.0.1:47132
I20260812 07:02:59.277397 20690 tablet_service.cc:1514] Processing DeleteTablet for tablet b074b9c44ffc4a1cb6e2851cf2697de0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:59 UTC) from {username='slave'} at 127.0.0.1:55738
I20260812 07:02:59.278270 20916 tablet_service.cc:1514] Processing DeleteTablet for tablet b074b9c44ffc4a1cb6e2851cf2697de0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:02:59 UTC) from {username='slave'} at 127.0.0.1:40552
I20260812 07:02:59.278610 21110 tablet_replica.cc:330] T b074b9c44ffc4a1cb6e2851cf2697de0 P a50e33df44ff474d9a0d4562b920e4f1: stopping tablet replica
I20260812 07:02:59.278625 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 20376
I20260812 07:02:59.279022 21110 raft_consensus.cc:2229] T b074b9c44ffc4a1cb6e2851cf2697de0 P a50e33df44ff474d9a0d4562b920e4f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:02:59.279506 21110 raft_consensus.cc:2258] T b074b9c44ffc4a1cb6e2851cf2697de0 P a50e33df44ff474d9a0d4562b920e4f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:59.280275 21111 tablet_replica.cc:330] T b074b9c44ffc4a1cb6e2851cf2697de0 P f2605dee56bb4b989c94602205a2e29e: stopping tablet replica
I20260812 07:02:59.280720 21111 raft_consensus.cc:2229] T b074b9c44ffc4a1cb6e2851cf2697de0 P f2605dee56bb4b989c94602205a2e29e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:02:59.281212 21111 raft_consensus.cc:2258] T b074b9c44ffc4a1cb6e2851cf2697de0 P f2605dee56bb4b989c94602205a2e29e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:59.283073 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:59.279919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47148 (local address 127.11.192.193:45323)
0812 07:02:59.280111 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:59.280117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:59.280650 (+ 533us) server_negotiation.cc:407] Connection header received
0812 07:02:59.280944 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:59.280949 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:59.281050 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:59.281123 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:59.282320 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:59.282326 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:59.282331 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:02:59.282350 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:59.282387 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:02:59.282402 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:59.282503 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:59.282748 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 07:02:59.282808 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:02:59.283429 21110 ts_tablet_manager.cc:1905] T b074b9c44ffc4a1cb6e2851cf2697de0 P a50e33df44ff474d9a0d4562b920e4f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:59.283439 21111 ts_tablet_manager.cc:1905] T b074b9c44ffc4a1cb6e2851cf2697de0 P f2605dee56bb4b989c94602205a2e29e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:59.284243 20576 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:59.290509 21111 ts_tablet_manager.cc:1918] T b074b9c44ffc4a1cb6e2851cf2697de0 P f2605dee56bb4b989c94602205a2e29e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:02:59.290751 21111 log.cc:1196] T b074b9c44ffc4a1cb6e2851cf2697de0 P f2605dee56bb4b989c94602205a2e29e: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/b074b9c44ffc4a1cb6e2851cf2697de0
I20260812 07:02:59.291320 21111 ts_tablet_manager.cc:1939] T b074b9c44ffc4a1cb6e2851cf2697de0 P f2605dee56bb4b989c94602205a2e29e: Deleting consensus metadata
I20260812 07:02:59.292992 21112 tablet_replica.cc:330] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3: stopping tablet replica
I20260812 07:02:59.293092 20252 catalog_manager.cc:4841] TS f2605dee56bb4b989c94602205a2e29e (127.11.192.194:46303): tablet b074b9c44ffc4a1cb6e2851cf2697de0 (table test-table [id=396133dc3bc048729b2634cd8dc8f90f]) successfully deleted
I20260812 07:02:59.293640 21112 raft_consensus.cc:2229] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:02:59.294289 21112 raft_consensus.cc:2258] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:02:59.296519 21112 ts_tablet_manager.cc:1905] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:02:59.301931 21112 ts_tablet_manager.cc:1918] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:02:59.302112 21112 log.cc:1196] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/b074b9c44ffc4a1cb6e2851cf2697de0
I20260812 07:02:59.302632 21112 ts_tablet_manager.cc:1939] T b074b9c44ffc4a1cb6e2851cf2697de0 P f395f23ebeff4dc2b3ddb55f871b75f3: Deleting consensus metadata
I20260812 07:02:59.303877 20252 catalog_manager.cc:4841] TS f395f23ebeff4dc2b3ddb55f871b75f3 (127.11.192.195:42115): tablet b074b9c44ffc4a1cb6e2851cf2697de0 (table test-table [id=396133dc3bc048729b2634cd8dc8f90f]) successfully deleted
I20260812 07:02:59.387970 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 20376
I20260812 07:02:59.410106 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 20586
W20260812 07:02:59.410106 20254 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.192.193:45323 (error 108)
I20260812 07:02:59.412037 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:59.411591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:45323 (local address 127.0.0.1:47160)
0812 07:02:59.411783 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:59.411905 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:45323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260812 07:02:59.412447 20254 catalog_manager.cc:4584] TS a50e33df44ff474d9a0d4562b920e4f1 (127.11.192.193:45323): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b074b9c44ffc4a1cb6e2851cf2697de0: Network error: Client connection negotiation failed: client connection to 127.11.192.193:45323: connect: Connection refused (error 111)
I20260812 07:02:59.413723 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:59.411246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55742 (local address 127.11.192.194:46303)
0812 07:02:59.411417 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:59.411422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:59.411438 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:02:59.411498 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:59.411503 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:59.411586 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:59.411679 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:59.413045 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:59.413050 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:59.413053 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:02:59.413071 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:59.413106 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:02:59.413118 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:59.413239 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:59.413350 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 07:02:59.413414 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:02:59.414417 20794 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:59.466025 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:59.465486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:45323 (local address 127.0.0.1:47168)
0812 07:02:59.465715 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:59.465827 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:45323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260812 07:02:59.467175 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:59.466758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:45323 (local address 127.0.0.1:47178)
0812 07:02:59.466931 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:59.467030 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:45323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 07:02:59.499660 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 20586
I20260812 07:02:59.525348 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 20809
I20260812 07:02:59.527571 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:59.527175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:45323 (local address 127.0.0.1:47182)
0812 07:02:59.527349 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:59.527438 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:45323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 07:02:59.528625 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:59.528268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:45323 (local address 127.0.0.1:47186)
0812 07:02:59.528407 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:59.528485 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:45323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 07:02:59.528774 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:59.526063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40578 (local address 127.11.192.195:42115)
0812 07:02:59.526252 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 07:02:59.526258 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:02:59.526349 (+ 91us) server_negotiation.cc:407] Connection header received
0812 07:02:59.526557 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:02:59.526562 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:02:59.526613 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:02:59.526706 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:02:59.528043 (+ 1337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:02:59.528050 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:02:59.528054 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:02:59.528073 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:02:59.528136 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:02:59.528157 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:02:59.528296 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 07:02:59.528436 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 07:02:59.528499 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 07:02:59.529429 21021 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:59.609333 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:59.608783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:45323 (local address 127.0.0.1:34686)
0812 07:02:59.609015 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:59.609122 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:45323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260812 07:02:59.610406 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:02:59.610009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:45323 (local address 127.0.0.1:34700)
0812 07:02:59.610215 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 07:02:59.610286 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:45323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260812 07:02:59.643527 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 20809
I20260812 07:02:59.670499 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 20227
I20260812 07:02:59.671581 20322 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:02:59.775437 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 20227
2026-08-12T07:02:59Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2618 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 07:02:59.821967 12035 mini_oidc.cc:152] Starting JWKS server
I20260812 07:02:59.824793 12035 webserver.cc:460] Webserver started at https://127.0.0.1:40873/ using document root <none> and password file <none>
I20260812 07:02:59.825067 12035 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 07:02:59.825690 12035 webserver.cc:460] Webserver started at http://127.0.0.1:37429/ using document root <none> and password file <none>
2026-08-12T07:02:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:02:59Z Disabled control of system clock
I20260812 07:02:59.855953 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:42675
--webserver_interface=127.11.192.254
--webserver_port=0
--builtin_ntp_servers=127.11.192.212:33285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:42675
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40873/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518136653111-12035-0/testchainca.cert with env {}
W20260812 07:03:00.170794 21136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:00.171097 21136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:00.171231 21136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:00.178328 21136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:03:00.178444 21136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:00.178509 21136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:00.178588 21136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:03:00.178653 21136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 07:03:00.178735 21136 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 07:03:00.178787 21136 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40873/jwks.json
I20260812 07:03:00.188313 21136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:33285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:42675
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:42675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40873/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518136653111-12035-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:00.189510 21136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:00.190747 21136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:00.201524 21148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:00.201615 21149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:00.201771 21152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:00.202574 21136 server_base.cc:1034] running on GCE node
I20260812 07:03:00.203230 21136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:00.204485 21136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:00.205695 21136 hybrid_clock.cc:648] HybridClock initialized: now 1786518180205645 us; error 35 us; skew 500 ppm
I20260812 07:03:00.206008 21136 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:03:00.211417 21136 webserver.cc:460] Webserver started at http://127.11.192.254:38831/ using document root <none> and password file <none>
I20260812 07:03:00.212066 21136 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:00.212211 21136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:00.212512 21136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:00.215340 21136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "0f9decfabcaf426aacb81e5fe6f679cd"
format_stamp: "Formatted at 2026-08-12 07:03:00 on dist-test-slave-lgr5"
server_key: "f7d838a219204b607aeef4b161de2b8e"
server_key_iv: "ce0e56223fc24363d694c3a522509f9b"
server_key_version: "encryptionkey@0"
I20260812 07:03:00.216135 21136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "0f9decfabcaf426aacb81e5fe6f679cd"
format_stamp: "Formatted at 2026-08-12 07:03:00 on dist-test-slave-lgr5"
server_key: "f7d838a219204b607aeef4b161de2b8e"
server_key_iv: "ce0e56223fc24363d694c3a522509f9b"
server_key_version: "encryptionkey@0"
I20260812 07:03:00.220980 21136 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 07:03:00.224396 21161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:00.225704 21136 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 07:03:00.225859 21136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "0f9decfabcaf426aacb81e5fe6f679cd"
format_stamp: "Formatted at 2026-08-12 07:03:00 on dist-test-slave-lgr5"
server_key: "f7d838a219204b607aeef4b161de2b8e"
server_key_iv: "ce0e56223fc24363d694c3a522509f9b"
server_key_version: "encryptionkey@0"
I20260812 07:03:00.226035 21136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:00.249233 21136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:00.249954 21136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:00.250206 21136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:00.266785 21136 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:42675
I20260812 07:03:00.266793 21251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:42675 every 8 connection(s)
I20260812 07:03:00.267987 21136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:03:00.271481 21252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:00.272095 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 21136
I20260812 07:03:00.272291 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:03:00.273114 12035 external_mini_cluster.cc:1428] Setting key ddf21288330a614a50c4de9b4bf401a4
I20260812 07:03:00.284897 21252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd: Bootstrap starting.
I20260812 07:03:00.286374 21258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:00.274998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43882 (local address 127.11.192.254:42675)
0812 07:03:00.275989 (+ 991us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:00.275999 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:00.276042 (+ 43us) server_negotiation.cc:407] Connection header received
0812 07:03:00.277683 (+ 1641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:00.277705 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:00.278140 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:00.278445 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:00.279039 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:00.280299 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:00.282151 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:00.282791 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:00.282923 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:00.284161 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:00.284173 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:00.284191 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:03:00.284745 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:00.284816 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:03:00.284836 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:00.284940 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:00.285182 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 07:03:00.285470 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":349,"threads_started":1}
I20260812 07:03:00.288003 21252 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:00.289145 21252 log.cc:824] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd: Log is configured to *not* fsync() on all Append() calls
I20260812 07:03:00.289739 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:03:00.291563 21252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd: No bootstrap required, opened a new log
I20260812 07:03:00.298692 21252 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9decfabcaf426aacb81e5fe6f679cd" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 42675 } }
I20260812 07:03:00.299023 21252 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:00.299115 21252 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f9decfabcaf426aacb81e5fe6f679cd, State: Initialized, Role: FOLLOWER
I20260812 07:03:00.299646 21252 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9decfabcaf426aacb81e5fe6f679cd" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 42675 } }
I20260812 07:03:00.299861 21252 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:03:00.299993 21252 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:03:00.300158 21252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:00.302333 21252 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9decfabcaf426aacb81e5fe6f679cd" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 42675 } }
I20260812 07:03:00.302791 21252 leader_election.cc:302] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f9decfabcaf426aacb81e5fe6f679cd; no voters:
I20260812 07:03:00.303320 21252 leader_election.cc:288] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:03:00.303468 21260 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:03:00.304343 21260 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd [term 1 LEADER]: Becoming Leader. State: Replica: 0f9decfabcaf426aacb81e5fe6f679cd, State: Running, Role: LEADER
I20260812 07:03:00.304888 21260 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9decfabcaf426aacb81e5fe6f679cd" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 42675 } }
I20260812 07:03:00.305310 21252 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:03:00.307480 21261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f9decfabcaf426aacb81e5fe6f679cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9decfabcaf426aacb81e5fe6f679cd" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 42675 } } }
I20260812 07:03:00.307488 21262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f9decfabcaf426aacb81e5fe6f679cd. Latest consensus state: current_term: 1 leader_uuid: "0f9decfabcaf426aacb81e5fe6f679cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9decfabcaf426aacb81e5fe6f679cd" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 42675 } } }
I20260812 07:03:00.307927 21262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:00.307927 21261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:00.309973 21273 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:03:00.314395 21273 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:03:00.321933 21273 catalog_manager.cc:1347] Generated new cluster ID: 1b1f59d358694c50998ce1db5eaa27df
I20260812 07:03:00.322032 21273 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:03:00.353425 21273 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:03:00.354584 21273 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:03:00.367208 21273 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f9decfabcaf426aacb81e5fe6f679cd: Generated new TSK 0
I20260812 07:03:00.367971 21273 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 07:03:00.407357 21308 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 07:03:00.407541 21308 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 07:03:00.408747 21258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:00.387333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43892 (local address 127.11.192.254:42675)
0812 07:03:00.387513 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:00.387522 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:00.387954 (+ 432us) server_negotiation.cc:407] Connection header received
0812 07:03:00.388312 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:00.388318 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:00.388403 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:00.388544 (+ 141us) server_negotiation.cc:226] Negotiated authn=JWT
0812 07:03:00.389011 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:00.390100 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:00.391467 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:00.392211 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:00.392339 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:00.392781 (+ 442us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 07:03:00.408133 (+ 15352us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40873/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 07:03:00.408452 (+ 319us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43892: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40873/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 07:03:00.408870 21258 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43892: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40873/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 07:03:00.410218 21307 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:00.387214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:42675 (local address 127.0.0.1:43892)
0812 07:03:00.387865 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:00.387880 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:00.388028 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:00.388558 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:00.388561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:00.388571 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0812 07:03:00.388878 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:00.388886 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:00.390216 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:00.390220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:00.391332 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:00.391343 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:00.392367 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:00.392372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:00.392486 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:00.392542 (+ 56us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 07:03:00.408355 (+ 15813us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40873/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 07:03:00.408584 (+ 229us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.192.254:42675: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40873/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":328,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.11.192.254:42675: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40873/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 07:03:00.412272 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 21136
I20260812 07:03:00.413331 21245 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:00.505570 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 21136
2026-08-12T07:03:00Z chronyd exiting
I20260812 07:03:00.562692 12035 test_util.cc:175] -----------------------------------------------
I20260812 07:03:00.562803 12035 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518136653111-12035-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (745 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 07:03:00.566473 12035 mini_oidc.cc:152] Starting JWKS server
I20260812 07:03:00.568244 12035 webserver.cc:460] Webserver started at https://127.0.0.1:45205/ using document root <none> and password file <none>
I20260812 07:03:00.568444 12035 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 07:03:00.569020 12035 webserver.cc:460] Webserver started at http://127.0.0.1:45167/ using document root <none> and password file <none>
2026-08-12T07:03:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:03:00Z Disabled control of system clock
I20260812 07:03:00.597998 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:33747
--webserver_interface=127.11.192.254
--webserver_port=0
--builtin_ntp_servers=127.11.192.212:35797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:33747
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45205/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518136653111-12035-0/testchainca.cert with env {}
W20260812 07:03:00.907238 21322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:00.907529 21322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:00.907666 21322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:00.914626 21322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:03:00.914731 21322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:00.914801 21322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:00.914880 21322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:03:00.914937 21322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 07:03:00.915017 21322 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 07:03:00.915082 21322 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45205/jwks.json
I20260812 07:03:00.924517 21322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:35797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:33747
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:33747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45205/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518136653111-12035-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:00.925635 21322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:00.926844 21322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:00.937826 21330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:00.937858 21333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:00.937979 21336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:00.938616 21322 server_base.cc:1034] running on GCE node
I20260812 07:03:00.939352 21322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:00.940701 21322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:00.941898 21322 hybrid_clock.cc:648] HybridClock initialized: now 1786518180941848 us; error 41 us; skew 500 ppm
I20260812 07:03:00.942214 21322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:03:00.944348 21322 webserver.cc:460] Webserver started at http://127.11.192.254:38671/ using document root <none> and password file <none>
I20260812 07:03:00.944977 21322 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:00.945081 21322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:00.945384 21322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:00.948063 21322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "89b08aeb0d3448c2aec60498abe96d84"
format_stamp: "Formatted at 2026-08-12 07:03:00 on dist-test-slave-lgr5"
server_key: "f0dfb959e07e750f98f7e78908757189"
server_key_iv: "219d552f14e8768ddd210ecd0ccd0154"
server_key_version: "encryptionkey@0"
I20260812 07:03:00.948882 21322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "89b08aeb0d3448c2aec60498abe96d84"
format_stamp: "Formatted at 2026-08-12 07:03:00 on dist-test-slave-lgr5"
server_key: "f0dfb959e07e750f98f7e78908757189"
server_key_iv: "219d552f14e8768ddd210ecd0ccd0154"
server_key_version: "encryptionkey@0"
I20260812 07:03:00.953569 21322 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:03:00.956771 21346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:00.957958 21322 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260812 07:03:00.958128 21322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "89b08aeb0d3448c2aec60498abe96d84"
format_stamp: "Formatted at 2026-08-12 07:03:00 on dist-test-slave-lgr5"
server_key: "f0dfb959e07e750f98f7e78908757189"
server_key_iv: "219d552f14e8768ddd210ecd0ccd0154"
server_key_version: "encryptionkey@0"
I20260812 07:03:00.958316 21322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:00.972021 21322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:00.972754 21322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:00.973019 21322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:00.989384 21322 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:33747
I20260812 07:03:00.989390 21444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:33747 every 8 connection(s)
I20260812 07:03:00.990612 21322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:03:00.991921 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 21322
I20260812 07:03:00.992173 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:03:00.993014 12035 external_mini_cluster.cc:1428] Setting key daf59373ca545f25b2ddcda3225f5ba3
I20260812 07:03:00.994782 21445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:01.005965 21449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:00.994938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43456 (local address 127.11.192.254:33747)
0812 07:03:00.995920 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:00.995930 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:00.995978 (+ 48us) server_negotiation.cc:407] Connection header received
0812 07:03:00.997518 (+ 1540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:00.997540 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:00.997954 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:00.998274 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:00.998786 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:01.000070 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:01.001805 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:01.002431 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:01.002551 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:01.003714 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:01.003725 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:01.003737 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:03:01.004349 (+ 612us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:01.004439 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:03:01.004460 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:01.004546 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:01.004773 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 07:03:01.005089 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":365,"threads_started":1}
I20260812 07:03:01.007365 21445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84: Bootstrap starting.
I20260812 07:03:01.009562 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:03:01.010468 21445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:01.011375 21445 log.cc:824] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84: Log is configured to *not* fsync() on all Append() calls
I20260812 07:03:01.014343 21445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84: No bootstrap required, opened a new log
I20260812 07:03:01.021509 21445 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b08aeb0d3448c2aec60498abe96d84" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33747 } }
I20260812 07:03:01.021867 21445 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:01.021958 21445 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89b08aeb0d3448c2aec60498abe96d84, State: Initialized, Role: FOLLOWER
I20260812 07:03:01.022485 21445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b08aeb0d3448c2aec60498abe96d84" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33747 } }
I20260812 07:03:01.022702 21445 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:03:01.022825 21445 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:03:01.022981 21445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:01.025146 21445 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b08aeb0d3448c2aec60498abe96d84" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33747 } }
I20260812 07:03:01.025594 21445 leader_election.cc:302] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89b08aeb0d3448c2aec60498abe96d84; no voters:
I20260812 07:03:01.026119 21445 leader_election.cc:288] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:03:01.026244 21451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:03:01.026875 21451 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84 [term 1 LEADER]: Becoming Leader. State: Replica: 89b08aeb0d3448c2aec60498abe96d84, State: Running, Role: LEADER
I20260812 07:03:01.027347 21451 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b08aeb0d3448c2aec60498abe96d84" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33747 } }
I20260812 07:03:01.027899 21445 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:03:01.030049 21456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89b08aeb0d3448c2aec60498abe96d84. Latest consensus state: current_term: 1 leader_uuid: "89b08aeb0d3448c2aec60498abe96d84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b08aeb0d3448c2aec60498abe96d84" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33747 } } }
I20260812 07:03:01.030230 21453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89b08aeb0d3448c2aec60498abe96d84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b08aeb0d3448c2aec60498abe96d84" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33747 } } }
I20260812 07:03:01.030424 21456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:01.030517 21453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:01.031908 21468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:03:01.036849 21468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:03:01.045140 21468 catalog_manager.cc:1347] Generated new cluster ID: 9d9386d382664d28a18da0cd5f2fa452
I20260812 07:03:01.045234 21468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:03:01.066007 21468 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:03:01.067132 21468 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:03:01.078755 21468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89b08aeb0d3448c2aec60498abe96d84: Generated new TSK 0
I20260812 07:03:01.079519 21468 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 07:03:01.112593 21500 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 07:03:01.112690 21500 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260812 07:03:01.113888 21499 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:01.094657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33747 (local address 127.0.0.1:43460)
0812 07:03:01.095226 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:01.095245 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:01.095357 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:01.095751 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:01.095754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:01.095762 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0812 07:03:01.095985 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:01.095994 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:01.097383 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:01.097387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:01.098408 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:01.098419 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:01.099343 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:01.099346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:01.099465 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:01.099504 (+ 39us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 07:03:01.113445 (+ 13941us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45205/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 07:03:01.113618 (+ 173us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.192.254:33747: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45205/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":323,"threads_started":1}
I20260812 07:03:01.113886 21449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:01.094772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43460 (local address 127.11.192.254:33747)
0812 07:03:01.094942 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:01.094949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:01.095328 (+ 379us) server_negotiation.cc:407] Connection header received
0812 07:03:01.095505 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:01.095512 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:01.095594 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:01.095756 (+ 162us) server_negotiation.cc:226] Negotiated authn=JWT
0812 07:03:01.096158 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:01.097268 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:01.098547 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:01.099214 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:01.099335 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:01.099749 (+ 414us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 07:03:01.113238 (+ 13489us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45205/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 07:03:01.113574 (+ 336us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43460: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45205/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":61}
W20260812 07:03:01.113994 21449 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43460: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45205/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260812 07:03:01.114908 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 21322
I20260812 07:03:01.116026 21437 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:01.207420 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 21322
2026-08-12T07:03:01Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (703 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 07:03:01.270115 12035 mini_oidc.cc:152] Starting JWKS server
I20260812 07:03:01.271906 12035 webserver.cc:460] Webserver started at https://127.0.0.1:44703/ using document root <none> and password file <none>
I20260812 07:03:01.272140 12035 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 07:03:01.272787 12035 webserver.cc:460] Webserver started at http://127.0.0.1:45239/ using document root <none> and password file <none>
2026-08-12T07:03:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:03:01Z Disabled control of system clock
I20260812 07:03:01.302742 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:33575
--webserver_interface=127.11.192.254
--webserver_port=0
--builtin_ntp_servers=127.11.192.212:32769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:33575
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44703/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 07:03:01.621305 21510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:01.621601 21510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:01.621776 21510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:01.628858 21510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:03:01.628976 21510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:01.629050 21510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:01.629139 21510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:03:01.629210 21510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 07:03:01.629310 21510 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 07:03:01.629371 21510 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44703/jwks.json
I20260812 07:03:01.638123 21510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:32769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:33575
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:33575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44703/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:01.639277 21510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:01.640992 21510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:01.652345 21515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:01.652403 21517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:01.652403 21523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:01.653133 21510 server_base.cc:1034] running on GCE node
I20260812 07:03:01.653772 21510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:01.655007 21510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:01.656175 21510 hybrid_clock.cc:648] HybridClock initialized: now 1786518181656124 us; error 43 us; skew 500 ppm
I20260812 07:03:01.656491 21510 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:03:01.658551 21510 webserver.cc:460] Webserver started at http://127.11.192.254:35237/ using document root <none> and password file <none>
I20260812 07:03:01.659155 21510 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:01.659240 21510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:01.659503 21510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:01.662182 21510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "0c236535dd874d10a82e7a84c24e0a8d"
format_stamp: "Formatted at 2026-08-12 07:03:01 on dist-test-slave-lgr5"
server_key: "110d58a3a0e0738160367d753393002a"
server_key_iv: "12b3c59b28485b42e6c9fcd3cb408938"
server_key_version: "encryptionkey@0"
I20260812 07:03:01.662873 21510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "0c236535dd874d10a82e7a84c24e0a8d"
format_stamp: "Formatted at 2026-08-12 07:03:01 on dist-test-slave-lgr5"
server_key: "110d58a3a0e0738160367d753393002a"
server_key_iv: "12b3c59b28485b42e6c9fcd3cb408938"
server_key_version: "encryptionkey@0"
I20260812 07:03:01.667593 21510 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260812 07:03:01.670727 21534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:01.671908 21510 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:03:01.672060 21510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "0c236535dd874d10a82e7a84c24e0a8d"
format_stamp: "Formatted at 2026-08-12 07:03:01 on dist-test-slave-lgr5"
server_key: "110d58a3a0e0738160367d753393002a"
server_key_iv: "12b3c59b28485b42e6c9fcd3cb408938"
server_key_version: "encryptionkey@0"
I20260812 07:03:01.672281 21510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:01.684692 21510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:01.685367 21510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:01.685617 21510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:01.702214 21510 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:33575
I20260812 07:03:01.702220 21622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:33575 every 8 connection(s)
I20260812 07:03:01.703403 21510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:03:01.706763 21623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:01.707444 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 21510
I20260812 07:03:01.707623 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:03:01.708303 12035 external_mini_cluster.cc:1428] Setting key 3b2772898aca59ab4a1c575f19b92a00
I20260812 07:03:01.719594 21623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d: Bootstrap starting.
I20260812 07:03:01.720978 21626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:01.710042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45806 (local address 127.11.192.254:33575)
0812 07:03:01.711004 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:01.711018 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:01.711058 (+ 40us) server_negotiation.cc:407] Connection header received
0812 07:03:01.712615 (+ 1557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:01.712637 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:01.713060 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:01.713357 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:01.713914 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:01.715117 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:01.716931 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:01.717643 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:01.717765 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:01.718888 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:01.718896 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:01.718907 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:03:01.719384 (+ 477us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:01.719454 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:03:01.719473 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:01.719562 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:01.719838 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 07:03:01.720153 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":354,"threads_started":1}
I20260812 07:03:01.722882 21623 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:01.723961 21623 log.cc:824] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d: Log is configured to *not* fsync() on all Append() calls
I20260812 07:03:01.724550 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:03:01.726375 21623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d: No bootstrap required, opened a new log
I20260812 07:03:01.733235 21623 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c236535dd874d10a82e7a84c24e0a8d" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33575 } }
I20260812 07:03:01.733600 21623 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:01.733687 21623 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c236535dd874d10a82e7a84c24e0a8d, State: Initialized, Role: FOLLOWER
I20260812 07:03:01.734180 21623 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c236535dd874d10a82e7a84c24e0a8d" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33575 } }
I20260812 07:03:01.734385 21623 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:03:01.734517 21623 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:03:01.734668 21623 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:01.736809 21623 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c236535dd874d10a82e7a84c24e0a8d" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33575 } }
I20260812 07:03:01.737247 21623 leader_election.cc:302] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c236535dd874d10a82e7a84c24e0a8d; no voters:
I20260812 07:03:01.737739 21623 leader_election.cc:288] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:03:01.737869 21629 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:03:01.738503 21629 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d [term 1 LEADER]: Becoming Leader. State: Replica: 0c236535dd874d10a82e7a84c24e0a8d, State: Running, Role: LEADER
I20260812 07:03:01.739018 21629 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c236535dd874d10a82e7a84c24e0a8d" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33575 } }
I20260812 07:03:01.739521 21623 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:03:01.742057 21630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c236535dd874d10a82e7a84c24e0a8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c236535dd874d10a82e7a84c24e0a8d" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33575 } } }
I20260812 07:03:01.742093 21633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c236535dd874d10a82e7a84c24e0a8d. Latest consensus state: current_term: 1 leader_uuid: "0c236535dd874d10a82e7a84c24e0a8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c236535dd874d10a82e7a84c24e0a8d" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33575 } } }
I20260812 07:03:01.742386 21630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:01.742452 21633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:01.743988 21638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:03:01.747620 21638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:03:01.756071 21638 catalog_manager.cc:1347] Generated new cluster ID: 3fdefa2f1c8e446180f69339737654c5
I20260812 07:03:01.756194 21638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:03:01.780607 21638 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:03:01.782279 21638 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:03:01.794461 21638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c236535dd874d10a82e7a84c24e0a8d: Generated new TSK 0
I20260812 07:03:01.795380 21638 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:03:01.818457 21626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:01.810816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45810 (local address 127.11.192.254:33575)
0812 07:03:01.811007 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:01.811015 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:01.811412 (+ 397us) server_negotiation.cc:407] Connection header received
0812 07:03:01.811630 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:01.811637 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:01.811700 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:01.811813 (+ 113us) server_negotiation.cc:226] Negotiated authn=JWT
0812 07:03:01.812360 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:01.813073 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:01.814987 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:01.815409 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:01.815508 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:01.816033 (+ 525us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 07:03:01.817866 (+ 1833us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44703/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 07:03:01.818174 (+ 308us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45810: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44703/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 07:03:01.818673 21670 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:01.810725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33575 (local address 127.0.0.1:45810)
0812 07:03:01.811312 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:01.811330 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:01.811475 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:01.811847 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:01.811853 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:01.811865 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0812 07:03:01.812222 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:01.812235 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:01.813207 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:01.813212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:01.814859 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:01.814875 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:01.815581 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:01.815588 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:01.815772 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:01.815816 (+ 44us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 07:03:01.818061 (+ 2245us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44703/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 07:03:01.818288 (+ 227us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.192.254:33575: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44703/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":300,"threads_started":1}
W20260812 07:03:01.818820 21626 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45810: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44703/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260812 07:03:01.819766 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 21510
I20260812 07:03:01.820801 21612 generic_service.cc:196] Checking for leaks (request via RPC)
E20260812 07:03:01.835197 21671 webserver.cc:533] Webserver: sslize failed:
E20260812 07:03:01.835294 21671 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 07:03:01.914001 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 21510
2026-08-12T07:03:01Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (704 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T07:03:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:03:01Z Disabled control of system clock
I20260812 07:03:02.005905 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:46469
--webserver_interface=127.11.192.254
--webserver_port=0
--builtin_ntp_servers=127.11.192.212:35513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:46469
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518136653111-12035-0/testchainca.cert with env {}
W20260812 07:03:02.324962 21683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:02.325249 21683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:02.325397 21683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:02.332499 21683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:03:02.332617 21683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:02.332690 21683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:02.332780 21683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:03:02.332845 21683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 07:03:02.332938 21683 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 07:03:02.332996 21683 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 07:03:02.342650 21683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:35513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:46469
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:46469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518136653111-12035-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:02.343832 21683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:02.345156 21683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:02.356457 21694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:02.356426 21698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:02.356498 21695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:02.357687 21683 server_base.cc:1034] running on GCE node
I20260812 07:03:02.358477 21683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:02.359742 21683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:02.360955 21683 hybrid_clock.cc:648] HybridClock initialized: now 1786518182360902 us; error 37 us; skew 500 ppm
I20260812 07:03:02.361316 21683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:03:02.363471 21683 webserver.cc:460] Webserver started at http://127.11.192.254:37733/ using document root <none> and password file <none>
I20260812 07:03:02.364171 21683 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:02.364284 21683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:02.364598 21683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:02.367396 21683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "26dcdc02afed401c963bc77f46f35faa"
format_stamp: "Formatted at 2026-08-12 07:03:02 on dist-test-slave-lgr5"
server_key: "813c8eba1bb4a4ae4de22ac7f69f8d77"
server_key_iv: "6760231bfad1e576892de31cc0d54cb7"
server_key_version: "encryptionkey@0"
I20260812 07:03:02.368259 21683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "26dcdc02afed401c963bc77f46f35faa"
format_stamp: "Formatted at 2026-08-12 07:03:02 on dist-test-slave-lgr5"
server_key: "813c8eba1bb4a4ae4de22ac7f69f8d77"
server_key_iv: "6760231bfad1e576892de31cc0d54cb7"
server_key_version: "encryptionkey@0"
I20260812 07:03:02.373193 21683 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 07:03:02.376502 21710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:02.377810 21683 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 07:03:02.377981 21683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "26dcdc02afed401c963bc77f46f35faa"
format_stamp: "Formatted at 2026-08-12 07:03:02 on dist-test-slave-lgr5"
server_key: "813c8eba1bb4a4ae4de22ac7f69f8d77"
server_key_iv: "6760231bfad1e576892de31cc0d54cb7"
server_key_version: "encryptionkey@0"
I20260812 07:03:02.378178 21683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:02.400777 21683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:02.401484 21683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:02.401726 21683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:02.418262 21683 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:46469
I20260812 07:03:02.418288 21798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:46469 every 8 connection(s)
I20260812 07:03:02.419466 21683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:03:02.422904 21799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:02.423090 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 21683
I20260812 07:03:02.423309 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:03:02.424161 12035 external_mini_cluster.cc:1428] Setting key ab16a490319e8e8467c800eddcb5a75d
I20260812 07:03:02.435642 21799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa: Bootstrap starting.
I20260812 07:03:02.437076 21807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:02.425954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52512 (local address 127.11.192.254:46469)
0812 07:03:02.426901 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:02.426910 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:02.426950 (+ 40us) server_negotiation.cc:407] Connection header received
0812 07:03:02.428842 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:02.428864 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:02.429280 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:02.429588 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:02.430140 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:02.431337 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:02.433089 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:02.433765 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:02.433882 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:02.435033 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:02.435050 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:02.435063 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:03:02.435541 (+ 478us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:02.435612 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:03:02.435635 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:02.435731 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:02.435940 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 07:03:02.436231 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":341,"threads_started":1}
I20260812 07:03:02.438616 21799 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:02.439663 21799 log.cc:824] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa: Log is configured to *not* fsync() on all Append() calls
I20260812 07:03:02.440533 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:03:02.442559 21799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa: No bootstrap required, opened a new log
I20260812 07:03:02.449630 21799 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26dcdc02afed401c963bc77f46f35faa" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46469 } }
I20260812 07:03:02.449916 21799 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:02.449999 21799 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26dcdc02afed401c963bc77f46f35faa, State: Initialized, Role: FOLLOWER
I20260812 07:03:02.450495 21799 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26dcdc02afed401c963bc77f46f35faa" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46469 } }
I20260812 07:03:02.450685 21799 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:03:02.450780 21799 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:03:02.450896 21799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:02.452854 21799 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26dcdc02afed401c963bc77f46f35faa" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46469 } }
I20260812 07:03:02.453311 21799 leader_election.cc:302] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26dcdc02afed401c963bc77f46f35faa; no voters:
I20260812 07:03:02.453789 21799 leader_election.cc:288] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:03:02.453905 21809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:03:02.454564 21809 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa [term 1 LEADER]: Becoming Leader. State: Replica: 26dcdc02afed401c963bc77f46f35faa, State: Running, Role: LEADER
I20260812 07:03:02.455075 21809 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26dcdc02afed401c963bc77f46f35faa" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46469 } }
I20260812 07:03:02.455528 21799 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:03:02.457636 21812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26dcdc02afed401c963bc77f46f35faa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26dcdc02afed401c963bc77f46f35faa" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46469 } } }
I20260812 07:03:02.457682 21816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26dcdc02afed401c963bc77f46f35faa. Latest consensus state: current_term: 1 leader_uuid: "26dcdc02afed401c963bc77f46f35faa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26dcdc02afed401c963bc77f46f35faa" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46469 } } }
I20260812 07:03:02.457995 21812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:02.458010 21816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:02.460625 21823 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:03:02.465276 21823 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:03:02.473409 21823 catalog_manager.cc:1347] Generated new cluster ID: cd2c58cc322c45169f35f615888eb82a
I20260812 07:03:02.473522 21823 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:03:02.508196 21823 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:03:02.509708 21823 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:03:02.516136 21823 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26dcdc02afed401c963bc77f46f35faa: Generated new TSK 0
I20260812 07:03:02.516889 21823 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:03:02.623135 21807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:02.534214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52516 (local address 127.11.192.254:46469)
0812 07:03:02.534417 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:02.534424 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:02.534918 (+ 494us) server_negotiation.cc:407] Connection header received
0812 07:03:02.535131 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:02.535135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:02.535203 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:02.535350 (+ 147us) server_negotiation.cc:226] Negotiated authn=JWT
0812 07:03:02.535856 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:02.536650 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:02.538678 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:02.539136 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:02.539240 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:02.539786 (+ 546us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 07:03:02.622517 (+ 82731us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 07:03:02.622882 (+ 365us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52516: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":60}
W20260812 07:03:02.623275 21807 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52516: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 07:03:02.623564 21857 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:02.534200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:46469 (local address 127.0.0.1:52516)
0812 07:03:02.534811 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:02.534829 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:02.534977 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:02.535360 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:02.535364 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:02.535378 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0812 07:03:02.535705 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:02.535718 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:02.536804 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:02.536809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:02.538537 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:02.538553 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:02.539299 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:02.539305 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:02.539496 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:02.539541 (+ 45us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 07:03:02.622814 (+ 83273us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 07:03:02.623144 (+ 330us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.192.254:46469: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":304,"threads_started":1}
I20260812 07:03:02.625013 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 21683
I20260812 07:03:02.626201 21791 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:02.725615 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 21683
2026-08-12T07:03:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (793 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (398 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (401 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:03:03 dist-test-slave-lgr5 krb5kdc[21887](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:03:03 dist-test-slave-lgr5 krb5kdc[21887](info): set up 2 sockets
Aug 12 07:03:03 dist-test-slave-lgr5 krb5kdc[21887](info): commencing operation
krb5kdc: starting...
W20260812 07:03:04.755657 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.003s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:03:04 dist-test-slave-lgr5 krb5kdc[21887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:03:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:03:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:03:04.957233 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:43301
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:34967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:43301
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:05.267089 21911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:05.267354 21911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:05.267480 21911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:05.274516 21911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:03:05.274619 21911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:05.274696 21911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:05.274768 21911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:03:05.274828 21911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:03:05.284287 21911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:34967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:43301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:43301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:05.285566 21911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:05.286814 21911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:05.297878 21918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:05.297851 21923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:05.297938 21921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:05.298957 21911 server_base.cc:1034] running on GCE node
I20260812 07:03:05.299641 21911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:05.300956 21911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:05.302171 21911 hybrid_clock.cc:648] HybridClock initialized: now 1786518185302115 us; error 43 us; skew 500 ppm
I20260812 07:03:05.302490 21911 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:05 dist-test-slave-lgr5 krb5kdc[21887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518185, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:05.310328 21911 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:03:05.311807 21911 webserver.cc:460] Webserver started at http://127.11.192.254:35785/ using document root <none> and password file <none>
I20260812 07:03:05.312525 21911 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:05.312644 21911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:05.312943 21911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:05.315830 21911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "2b6f144ec28c4a55ba425dd95f3b2b41"
format_stamp: "Formatted at 2026-08-12 07:03:05 on dist-test-slave-lgr5"
server_key: "c5d15a1cedee48b070c7eb7127079f27"
server_key_iv: "8cb11ee062619cc989a46adef688b2d5"
server_key_version: "encryptionkey@0"
I20260812 07:03:05.316635 21911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "2b6f144ec28c4a55ba425dd95f3b2b41"
format_stamp: "Formatted at 2026-08-12 07:03:05 on dist-test-slave-lgr5"
server_key: "c5d15a1cedee48b070c7eb7127079f27"
server_key_iv: "8cb11ee062619cc989a46adef688b2d5"
server_key_version: "encryptionkey@0"
I20260812 07:03:05.321470 21911 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 07:03:05.324915 21937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:05.326162 21911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:03:05.326321 21911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "2b6f144ec28c4a55ba425dd95f3b2b41"
format_stamp: "Formatted at 2026-08-12 07:03:05 on dist-test-slave-lgr5"
server_key: "c5d15a1cedee48b070c7eb7127079f27"
server_key_iv: "8cb11ee062619cc989a46adef688b2d5"
server_key_version: "encryptionkey@0"
I20260812 07:03:05.326488 21911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:05.341575 21911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:05.345170 21911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:05.345405 21911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:05.362116 21911 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:43301
I20260812 07:03:05.362138 22027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:43301 every 8 connection(s)
I20260812 07:03:05.363308 21911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:03:05.366708 22028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:05.372752 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 21911
I20260812 07:03:05.372954 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:03:05.373555 12035 external_mini_cluster.cc:1428] Setting key effb7036c7c4629a5aedc15b0d2db50d
I20260812 07:03:05.376675 22028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41: Bootstrap starting.
I20260812 07:03:05.379952 22028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:05.381052 22028 log.cc:824] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41: Log is configured to *not* fsync() on all Append() calls
I20260812 07:03:05.383903 22028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41: No bootstrap required, opened a new log
Aug 12 07:03:05 dist-test-slave-lgr5 krb5kdc[21887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:03:05.392230 22028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6f144ec28c4a55ba425dd95f3b2b41" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43301 } }
I20260812 07:03:05.392614 22028 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:05.392727 22028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6f144ec28c4a55ba425dd95f3b2b41, State: Initialized, Role: FOLLOWER
I20260812 07:03:05.393258 22028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6f144ec28c4a55ba425dd95f3b2b41" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43301 } }
I20260812 07:03:05.393529 22028 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:03:05.393672 22028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:03:05.393868 22028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:05.395398 22031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:05.375330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48660 (local address 127.11.192.254:43301)
0812 07:03:05.376110 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:05.376126 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:05.376167 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:03:05.377839 (+ 1672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:05.377862 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:05.378226 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:05.378516 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:05.380310 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:05.381732 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:05.383324 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:05.383961 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:05.384174 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:05.388296 (+ 4122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:05.388304 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:05.388317 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:05.388364 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:05.391318 (+ 2954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:05.392138 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:05.392144 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:05.392150 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:05.392307 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:05.392702 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:05.392707 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:05.392710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:05.393307 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:05.393558 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:05.394001 (+ 443us) server_negotiation.cc:299] Negotiation successful
0812 07:03:05.394330 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":325,"threads_started":1}
I20260812 07:03:05.396307 22028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6f144ec28c4a55ba425dd95f3b2b41" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43301 } }
I20260812 07:03:05.396781 22028 leader_election.cc:302] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b6f144ec28c4a55ba425dd95f3b2b41; no voters:
I20260812 07:03:05.397370 22028 leader_election.cc:288] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:03:05.397476 22035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:03:05.398358 22035 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6f144ec28c4a55ba425dd95f3b2b41, State: Running, Role: LEADER
I20260812 07:03:05.398904 22035 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6f144ec28c4a55ba425dd95f3b2b41" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43301 } }
I20260812 07:03:05.399756 22028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:03:05.401818 22036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b6f144ec28c4a55ba425dd95f3b2b41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6f144ec28c4a55ba425dd95f3b2b41" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43301 } } }
I20260812 07:03:05.402230 22036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:05.401788 22040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b6f144ec28c4a55ba425dd95f3b2b41. Latest consensus state: current_term: 1 leader_uuid: "2b6f144ec28c4a55ba425dd95f3b2b41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6f144ec28c4a55ba425dd95f3b2b41" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43301 } } }
I20260812 07:03:05.402419 22040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:05.403431 22046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:03:05.408463 22046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:03:05.416723 22046 catalog_manager.cc:1347] Generated new cluster ID: da724314c6024c2c80374f6888b3f298
I20260812 07:03:05.416815 22046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:03:05.425169 22046 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:03:05.426271 22046 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:03:05.433214 22046 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b6f144ec28c4a55ba425dd95f3b2b41: Generated new TSK 0
I20260812 07:03:05.433885 22046 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
I20260812 07:03:05.501318 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43301
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:34967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:05.879176 22075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:05.879441 22075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:05.879547 22075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:05.887558 22075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:05.887691 22075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:05.887828 22075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:03:05.897610 22075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:34967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:05.898729 22075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:05.900015 22075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:05.912846 22089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:05.912940 22085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:05.912901 22084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:05.913956 22075 server_base.cc:1034] running on GCE node
I20260812 07:03:05.914539 22075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:05.915443 22075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:05.916648 22075 hybrid_clock.cc:648] HybridClock initialized: now 1786518185916593 us; error 40 us; skew 500 ppm
I20260812 07:03:05.916956 22075 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:05 dist-test-slave-lgr5 krb5kdc[21887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518185, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:05.919940 22075 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:03:05.921330 22075 webserver.cc:460] Webserver started at http://127.11.192.193:34571/ using document root <none> and password file <none>
I20260812 07:03:05.921983 22075 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:05.922109 22075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:05.922420 22075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:05.925318 22075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "407ee7c0467145d39621f2a3250c8353"
format_stamp: "Formatted at 2026-08-12 07:03:05 on dist-test-slave-lgr5"
server_key: "ca8f2b3eb96f1202efd5c1938e1d197e"
server_key_iv: "a4c4142edd41e8b952034f6f25fb82e6"
server_key_version: "encryptionkey@0"
I20260812 07:03:05.926180 22075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "407ee7c0467145d39621f2a3250c8353"
format_stamp: "Formatted at 2026-08-12 07:03:05 on dist-test-slave-lgr5"
server_key: "ca8f2b3eb96f1202efd5c1938e1d197e"
server_key_iv: "a4c4142edd41e8b952034f6f25fb82e6"
server_key_version: "encryptionkey@0"
I20260812 07:03:05.931269 22075 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260812 07:03:05.934572 22103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:05.935811 22075 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:03:05.935976 22075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "407ee7c0467145d39621f2a3250c8353"
format_stamp: "Formatted at 2026-08-12 07:03:05 on dist-test-slave-lgr5"
server_key: "ca8f2b3eb96f1202efd5c1938e1d197e"
server_key_iv: "a4c4142edd41e8b952034f6f25fb82e6"
server_key_version: "encryptionkey@0"
I20260812 07:03:05.936209 22075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:05.946746 22075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:05.950435 22075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:05.950713 22075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:05.951601 22075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:05.952980 22075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:05.953068 22075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:05.953199 22075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:05.953260 22075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:05.988610 22075 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:42541
I20260812 07:03:05.988653 22321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:42541 every 8 connection(s)
I20260812 07:03:05.989784 22075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:03:05.994356 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 22075
I20260812 07:03:05.994572 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:03:05.995383 12035 external_mini_cluster.cc:1428] Setting key e0a5011493453828c5ffebb9a4373354
Aug 12 07:03:05 dist-test-slave-lgr5 krb5kdc[21887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518185, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:03:06.010267 22329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:05.992054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:50915 (local address 127.11.192.254:43301)
0812 07:03:05.992505 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:05.992510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:05.993337 (+ 827us) server_negotiation.cc:407] Connection header received
0812 07:03:05.995674 (+ 2337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:05.995679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:05.995758 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:05.995851 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:05.997606 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:05.998306 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:06.000468 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:06.000897 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:06.001008 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:06.004906 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:06.004911 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:06.004915 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:06.004948 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:06.007386 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:06.008060 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:06.008066 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:06.008069 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:06.008246 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:06.008647 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:06.008651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:06.008653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:06.008971 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:06.009111 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:06.009722 (+ 611us) server_negotiation.cc:299] Negotiation successful
0812 07:03:06.009934 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":218,"threads_started":1}
I20260812 07:03:06.010917 22331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:05.992514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43301 (local address 127.11.192.193:50915)
0812 07:03:05.993174 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:05.993218 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:05.995360 (+ 2142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:05.996122 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:05.996134 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:05.996743 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:05.997408 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:05.997434 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:05.998498 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:05.998504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:06.000316 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:06.000359 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:06.001114 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:06.001120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:06.001374 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:06.002043 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:06.002070 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:06.004676 (+ 2606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:06.007538 (+ 2862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:06.007545 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:06.007555 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:06.007940 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:06.008347 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:06.008351 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:06.008353 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:06.008532 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:06.009070 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:06.009076 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:06.009343 (+ 267us) client_negotiation.cc:769] Sending connection context
0812 07:03:06.009563 (+ 220us) client_negotiation.cc:240] Negotiation successful
0812 07:03:06.009914 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":255,"threads_started":1}
I20260812 07:03:06.012328 22327 heartbeater.cc:347] Connected to a master server at 127.11.192.254:43301
I20260812 07:03:06.012604 22327 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:06.013266 22327 heartbeater.cc:511] Master 127.11.192.254:43301 requested a full tablet report, sending...
I20260812 07:03:06.015224 21964 ts_manager.cc:194] Registered new tserver with Master: 407ee7c0467145d39621f2a3250c8353 (127.11.192.193:42541)
I20260812 07:03:06.016719 21964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:50915
WARNING: no policy specified for kudu/127.11.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
I20260812 07:03:06.044551 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43301
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:34967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:06.353648 22334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:06.353919 22334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:06.354044 22334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:06.361068 22334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:06.361176 22334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:06.361327 22334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:03:06.371021 22334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:34967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:06.372198 22334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:06.373508 22334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:06.386287 22345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:06.386405 22348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:06.386513 22342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:06.387233 22334 server_base.cc:1034] running on GCE node
I20260812 07:03:06.387846 22334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:06.388810 22334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:06.390021 22334 hybrid_clock.cc:648] HybridClock initialized: now 1786518186389963 us; error 42 us; skew 500 ppm
I20260812 07:03:06.390338 22334 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:06 dist-test-slave-lgr5 krb5kdc[21887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518186, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:06.393405 22334 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:03:06.394714 22334 webserver.cc:460] Webserver started at http://127.11.192.194:41599/ using document root <none> and password file <none>
I20260812 07:03:06.395350 22334 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:06.395464 22334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:06.395761 22334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:06.398537 22334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "23d2477737184b989e8b3f03fd20fb28"
format_stamp: "Formatted at 2026-08-12 07:03:06 on dist-test-slave-lgr5"
server_key: "9c9580e92a27ce4fe2318f7ce6606fc4"
server_key_iv: "82b66dfb02db3c8624269f61eb91d986"
server_key_version: "encryptionkey@0"
I20260812 07:03:06.399271 22334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "23d2477737184b989e8b3f03fd20fb28"
format_stamp: "Formatted at 2026-08-12 07:03:06 on dist-test-slave-lgr5"
server_key: "9c9580e92a27ce4fe2318f7ce6606fc4"
server_key_iv: "82b66dfb02db3c8624269f61eb91d986"
server_key_version: "encryptionkey@0"
I20260812 07:03:06.404294 22334 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260812 07:03:06.407524 22363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:06.408748 22334 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:03:06.408887 22334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "23d2477737184b989e8b3f03fd20fb28"
format_stamp: "Formatted at 2026-08-12 07:03:06 on dist-test-slave-lgr5"
server_key: "9c9580e92a27ce4fe2318f7ce6606fc4"
server_key_iv: "82b66dfb02db3c8624269f61eb91d986"
server_key_version: "encryptionkey@0"
I20260812 07:03:06.409060 22334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:06.425334 22334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:06.429333 22334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:06.429663 22334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:06.430549 22334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:06.431789 22334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:06.431859 22334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:06.431951 22334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:06.431993 22334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:06.460206 22334 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:46869
I20260812 07:03:06.460235 22587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:46869 every 8 connection(s)
I20260812 07:03:06.461370 22334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:03:06.470541 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 22334
I20260812 07:03:06.470744 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:03:06.471269 12035 external_mini_cluster.cc:1428] Setting key b6bfaac3000de465c81ba556cc4a45ee
Aug 12 07:03:06 dist-test-slave-lgr5 krb5kdc[21887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518186, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:03:06.480468 22329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:06.463700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:47417 (local address 127.11.192.254:43301)
0812 07:03:06.463909 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:06.463916 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:06.465045 (+ 1129us) server_negotiation.cc:407] Connection header received
0812 07:03:06.466757 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:06.466762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:06.466833 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:06.466919 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:06.468310 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:06.469000 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:06.470536 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:06.471162 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:06.471288 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:06.474643 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:06.474649 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:06.474655 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:06.474695 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:06.477611 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:06.478281 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:06.478286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:06.478289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:06.478445 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:06.478853 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:06.478857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:06.478860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:06.479180 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:06.479315 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:06.479924 (+ 609us) server_negotiation.cc:299] Negotiation successful
0812 07:03:06.480165 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:03:06.481309 22591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:06.464198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43301 (local address 127.11.192.194:47417)
0812 07:03:06.464893 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:06.464929 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:06.466489 (+ 1560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:06.467098 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:06.467105 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:06.467595 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:06.468171 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:06.468192 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:06.469126 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:06.469130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:06.470361 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:06.470375 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:06.471367 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:06.471375 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:06.471602 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:06.472239 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:06.472261 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:06.474390 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:06.477735 (+ 3345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:06.477741 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:06.477752 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:06.478154 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:06.478559 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:06.478562 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:06.478565 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:06.478748 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:06.479289 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:06.479294 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:06.479585 (+ 291us) client_negotiation.cc:769] Sending connection context
0812 07:03:06.479798 (+ 213us) client_negotiation.cc:240] Negotiation successful
0812 07:03:06.480191 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":271,"threads_started":1}
I20260812 07:03:06.482714 22589 heartbeater.cc:347] Connected to a master server at 127.11.192.254:43301
I20260812 07:03:06.483003 22589 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:06.483676 22589 heartbeater.cc:511] Master 127.11.192.254:43301 requested a full tablet report, sending...
I20260812 07:03:06.485105 21964 ts_manager.cc:194] Registered new tserver with Master: 23d2477737184b989e8b3f03fd20fb28 (127.11.192.194:46869)
I20260812 07:03:06.485865 21964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:47417
WARNING: no policy specified for kudu/127.11.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
I20260812 07:03:06.520310 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43301
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:34967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:06.829572 22598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:06.829854 22598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:06.829999 22598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:06.837086 22598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:06.837193 22598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:06.837342 22598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:03:06.846495 22598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:34967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:06.847574 22598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:06.848825 22598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:06.860724 22608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:06.860800 22614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:06.860774 22611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:06.861754 22598 server_base.cc:1034] running on GCE node
I20260812 07:03:06.862298 22598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:06.863128 22598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:06.864320 22598 hybrid_clock.cc:648] HybridClock initialized: now 1786518186864275 us; error 33 us; skew 500 ppm
I20260812 07:03:06.864616 22598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:06 dist-test-slave-lgr5 krb5kdc[21887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518186, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:06.867532 22598 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:03:06.868927 22598 webserver.cc:460] Webserver started at http://127.11.192.195:37911/ using document root <none> and password file <none>
I20260812 07:03:06.869586 22598 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:06.869700 22598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:06.869989 22598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:06.873015 22598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "3578024066804963854ecbd251cb1bf4"
format_stamp: "Formatted at 2026-08-12 07:03:06 on dist-test-slave-lgr5"
server_key: "ea3e8f73f2c13d7ae232ce2708e7e5c7"
server_key_iv: "72d009103279a421075d70196f235dfd"
server_key_version: "encryptionkey@0"
I20260812 07:03:06.873766 22598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "3578024066804963854ecbd251cb1bf4"
format_stamp: "Formatted at 2026-08-12 07:03:06 on dist-test-slave-lgr5"
server_key: "ea3e8f73f2c13d7ae232ce2708e7e5c7"
server_key_iv: "72d009103279a421075d70196f235dfd"
server_key_version: "encryptionkey@0"
I20260812 07:03:06.878603 22598 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:03:06.881816 22627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:06.882982 22598 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:03:06.883123 22598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "3578024066804963854ecbd251cb1bf4"
format_stamp: "Formatted at 2026-08-12 07:03:06 on dist-test-slave-lgr5"
server_key: "ea3e8f73f2c13d7ae232ce2708e7e5c7"
server_key_iv: "72d009103279a421075d70196f235dfd"
server_key_version: "encryptionkey@0"
I20260812 07:03:06.883306 22598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:06.909860 22598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:06.914057 22598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:06.914324 22598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:06.915244 22598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:06.916556 22598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:06.916630 22598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:06.916739 22598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:06.916788 22598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:06.946422 22598 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:35231
I20260812 07:03:06.946491 22840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:35231 every 8 connection(s)
I20260812 07:03:06.947606 22598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:03:06.957422 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 22598
I20260812 07:03:06.957623 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:03:06.958073 12035 external_mini_cluster.cc:1428] Setting key c014a559d8eb1750c818e40d22cdcfed
Aug 12 07:03:06 dist-test-slave-lgr5 krb5kdc[21887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518186, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:03:06.966732 22329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:06.949893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:54389 (local address 127.11.192.254:43301)
0812 07:03:06.950060 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:06.950064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:06.951159 (+ 1095us) server_negotiation.cc:407] Connection header received
0812 07:03:06.952959 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:06.952965 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:06.953095 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:06.953187 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:06.954620 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:06.955363 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:06.956901 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:06.957457 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:06.957598 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:06.961338 (+ 3740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:06.961343 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:06.961347 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:06.961379 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:06.963820 (+ 2441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:06.964550 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:06.964554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:06.964556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:06.964681 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:06.965096 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:06.965100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:06.965102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:06.965371 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:06.965523 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:06.966166 (+ 643us) server_negotiation.cc:299] Negotiation successful
0812 07:03:06.966392 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:03:06.967514 22845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:06.950316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43301 (local address 127.11.192.195:54389)
0812 07:03:06.950983 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:06.951044 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:06.952694 (+ 1650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:06.953435 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:06.953442 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:06.953912 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:06.954453 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:06.954470 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:06.955507 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:06.955511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:06.956763 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:06.956776 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:06.957698 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:06.957706 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:06.957917 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:06.958526 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:06.958556 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:06.961103 (+ 2547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:06.963950 (+ 2847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:06.963958 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:06.963981 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:06.964427 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:06.964783 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:06.964788 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:06.964792 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:06.964981 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:06.965496 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:06.965503 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:06.965799 (+ 296us) client_negotiation.cc:769] Sending connection context
0812 07:03:06.966026 (+ 227us) client_negotiation.cc:240] Negotiation successful
0812 07:03:06.966481 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":268,"threads_started":1}
I20260812 07:03:06.968678 22843 heartbeater.cc:347] Connected to a master server at 127.11.192.254:43301
I20260812 07:03:06.968940 22843 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:06.969583 22843 heartbeater.cc:511] Master 127.11.192.254:43301 requested a full tablet report, sending...
I20260812 07:03:06.970867 21964 ts_manager.cc:194] Registered new tserver with Master: 3578024066804963854ecbd251cb1bf4 (127.11.192.195:35231)
I20260812 07:03:06.971644 21964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:54389
I20260812 07:03:06.973697 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/test-admin.keytab.
Aug 12 07:03:06 dist-test-slave-lgr5 krb5kdc[21887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:07.002859 12035 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 07:03:07.007648 22857 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:07.005386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36688 (local address 127.11.192.193:42541)
0812 07:03:07.005989 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:07.005996 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:07.006023 (+ 27us) server_negotiation.cc:407] Connection header received
0812 07:03:07.006082 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:07.006088 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:07.006279 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:07.006402 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:07.007284 (+ 882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36688: BlockingRecv error: recv got EOF from 127.0.0.1:36688 (error 108)
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":324,"threads_started":1}
W20260812 07:03:07.007740 12035 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/krb5cc' of type FILE
Aug 12 07:03:07 dist-test-slave-lgr5 krb5kdc[21887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:07.009425 12035 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 07:03:07 dist-test-slave-lgr5 krb5kdc[21887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:07.010660 12035 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 07:03:07 dist-test-slave-lgr5 krb5kdc[21887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.193@KRBTEST.COM
I20260812 07:03:07.019323 22327 heartbeater.cc:503] Master 127.11.192.254:43301 was elected leader, sending a full tablet report...
I20260812 07:03:07.024643 22857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:07.012038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36698 (local address 127.11.192.193:42541)
0812 07:03:07.012256 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:07.012261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:07.012452 (+ 191us) server_negotiation.cc:407] Connection header received
0812 07:03:07.012594 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:07.012598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:07.012648 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:07.012715 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:07.013731 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:07.014650 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:07.015747 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:07.016334 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:07.016427 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:07.019523 (+ 3096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:07.019537 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:07.019551 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:07.019586 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:07.022357 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:07.022959 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:07.022964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:07.022966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:07.023087 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:07.023348 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:07.023351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:07.023353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:07.023678 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:07.023882 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:07.024167 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 07:03:07.024395 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
W20260812 07:03:07.029613 22857 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:07.028124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36708 (local address 127.11.192.193:42541)
0812 07:03:07.028292 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:07.028298 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:07.028447 (+ 149us) server_negotiation.cc:407] Connection header received
0812 07:03:07.028591 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:07.028596 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:07.028661 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:07.028733 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:07.029397 (+ 664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36708: BlockingRecv error: recv got EOF from 127.0.0.1:36708 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260812 07:03:07.029753 12035 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/krb5cc' of type FILE
Aug 12 07:03:07 dist-test-slave-lgr5 krb5kdc[21887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:07.031092 12035 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 07:03:07 dist-test-slave-lgr5 krb5kdc[21887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:07.032286 12035 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 07:03:07 dist-test-slave-lgr5 krb5kdc[21887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.193@KRBTEST.COM
I20260812 07:03:07.044772 22857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:07.033344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36710 (local address 127.11.192.193:42541)
0812 07:03:07.033506 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:07.033511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:07.033762 (+ 251us) server_negotiation.cc:407] Connection header received
0812 07:03:07.033888 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:07.033893 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:07.033948 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:07.034020 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:07.034873 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:07.035540 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:07.036726 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:07.037158 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:07.037263 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:07.040336 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:07.040341 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:07.040344 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:07.040372 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:07.042517 (+ 2145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:07.043078 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:07.043083 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:07.043087 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:07.043202 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:07.043542 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:07.043547 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:07.043551 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:07.043791 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:07.044023 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:07.044357 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 07:03:07.044548 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 07:03:07.048530 22857 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:07.047030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36724 (local address 127.11.192.193:42541)
0812 07:03:07.047185 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:07.047190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:07.047434 (+ 244us) server_negotiation.cc:407] Connection header received
0812 07:03:07.047559 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:07.047564 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:07.047628 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:07.047700 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:07.048328 (+ 628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36724: BlockingRecv error: recv got EOF from 127.0.0.1:36724 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260812 07:03:07.048694 12035 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518136653111-12035-0/krb5kdc/krb5cc' of type FILE
Aug 12 07:03:07 dist-test-slave-lgr5 krb5kdc[21887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:07.049916 12035 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 07:03:07 dist-test-slave-lgr5 krb5kdc[21887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:07.051050 12035 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 07:03:07 dist-test-slave-lgr5 krb5kdc[21887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.193@KRBTEST.COM
I20260812 07:03:07.063315 22857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:07.052114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36732 (local address 127.11.192.193:42541)
0812 07:03:07.052316 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:07.052323 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:07.052485 (+ 162us) server_negotiation.cc:407] Connection header received
0812 07:03:07.052579 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:07.052582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:07.052633 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:07.052707 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:07.053520 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:07.054207 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:07.055434 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:07.055859 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:07.055956 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:07.058883 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:07.058888 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:07.058891 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:07.058920 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:07.061241 (+ 2321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:07.061801 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:07.061805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:07.061807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:07.061943 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:07.062233 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:07.062235 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:07.062237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:07.062504 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:07.062737 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:07.062840 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 07:03:07.063003 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:03:07.064604 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 22075
I20260812 07:03:07.075176 22857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:07.065333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36744 (local address 127.11.192.193:42541)
0812 07:03:07.065495 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:07.065500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:07.065650 (+ 150us) server_negotiation.cc:407] Connection header received
0812 07:03:07.065770 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:07.065773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:07.065818 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:07.065884 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:07.066755 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:07.067373 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:07.068573 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:07.068982 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:07.069085 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:07.070908 (+ 1823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:07.070912 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:07.070915 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:07.070941 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:07.073068 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:07.073668 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:07.073671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:07.073673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:07.073779 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:07.074110 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:07.074113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:07.074115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:07.074325 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:07.074499 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:07.074731 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 07:03:07.074867 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:03:07.075742 22311 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:07.157897 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 22075
I20260812 07:03:07.182199 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 22334
Aug 12 07:03:07 dist-test-slave-lgr5 krb5kdc[21887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.194@KRBTEST.COM
I20260812 07:03:07.196388 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:07.183081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38194 (local address 127.11.192.194:46869)
0812 07:03:07.183588 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:07.183595 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:07.183622 (+ 27us) server_negotiation.cc:407] Connection header received
0812 07:03:07.183686 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:07.183692 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:07.183909 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:07.184037 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:07.184966 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:07.186015 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:07.187253 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:07.187804 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:07.187920 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:07.191361 (+ 3441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:07.191372 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:07.191387 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:07.191419 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:07.193973 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:07.194577 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:07.194585 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:07.194588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:07.194717 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:07.195011 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:07.195014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:07.195016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:07.195340 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:07.195605 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:07.195793 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 07:03:07.196047 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":271,"threads_started":1}
I20260812 07:03:07.197595 22577 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:07.277681 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 22334
I20260812 07:03:07.296223 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 22598
Aug 12 07:03:07 dist-test-slave-lgr5 krb5kdc[21887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.195@KRBTEST.COM
I20260812 07:03:07.310850 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:07.297094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32854 (local address 127.11.192.195:35231)
0812 07:03:07.297649 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:07.297656 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:07.297684 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:03:07.297744 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:07.297752 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:07.297972 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:07.298098 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:07.299090 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:07.300178 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:07.301447 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:07.301983 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:07.302075 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:07.306006 (+ 3931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:07.306017 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:07.306031 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:07.306068 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:07.308596 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:07.309177 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:07.309183 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:07.309185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:07.309301 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:07.309576 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:07.309579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:07.309581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:07.309915 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:07.310148 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:07.310324 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 07:03:07.310530 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":268,"threads_started":1}
I20260812 07:03:07.312096 22834 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:07.392047 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 22598
I20260812 07:03:07.417752 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 21911
I20260812 07:03:07.418938 22023 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:07.519806 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 21911
2026-08-12T07:03:07Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3995 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:03:07 dist-test-slave-lgr5 krb5kdc[22890](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:03:07 dist-test-slave-lgr5 krb5kdc[22890](info): set up 2 sockets
Aug 12 07:03:07 dist-test-slave-lgr5 krb5kdc[22890](info): commencing operation
krb5kdc: starting...
W20260812 07:03:08.768430 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:03:08 dist-test-slave-lgr5 krb5kdc[22890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:03:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:03:08Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.254.keytab.
Entry for principal oryx/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.254.keytab.
I20260812 07:03:08.971114 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:45769
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.254.keytab
--principal=oryx/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:36469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:45769
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:09.291692 22909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:09.291954 22909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:09.292057 22909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:09.299006 22909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:03:09.299103 22909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:09.299160 22909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:09.299214 22909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:03:09.299257 22909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:03:09.308502 22909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:36469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:45769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.254.keytab
--principal=oryx/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:45769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:09.309626 22909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:09.310804 22909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:09.321641 22915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:09.321720 22914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:09.321688 22918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:09.322460 22909 server_base.cc:1034] running on GCE node
I20260812 07:03:09.323108 22909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:09.324386 22909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:09.325591 22909 hybrid_clock.cc:648] HybridClock initialized: now 1786518189325549 us; error 39 us; skew 500 ppm
I20260812 07:03:09.325902 22909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:09 dist-test-slave-lgr5 krb5kdc[22890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518189, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:09.328814 22909 init.cc:377] Logged in from keytab as oryx/127.11.192.254@KRBTEST.COM (short username oryx)
I20260812 07:03:09.330169 22909 webserver.cc:460] Webserver started at http://127.11.192.254:40075/ using document root <none> and password file <none>
I20260812 07:03:09.330811 22909 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:09.330932 22909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:09.331231 22909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:09.334069 22909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "f729c2efda9346f2a36c4ba13c5c56f9"
format_stamp: "Formatted at 2026-08-12 07:03:09 on dist-test-slave-lgr5"
server_key: "0c9cf76243513f4bf01a0b8271c91ec4"
server_key_iv: "98347f0d23fca03f1b71916735cd2a72"
server_key_version: "encryptionkey@0"
I20260812 07:03:09.334785 22909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "f729c2efda9346f2a36c4ba13c5c56f9"
format_stamp: "Formatted at 2026-08-12 07:03:09 on dist-test-slave-lgr5"
server_key: "0c9cf76243513f4bf01a0b8271c91ec4"
server_key_iv: "98347f0d23fca03f1b71916735cd2a72"
server_key_version: "encryptionkey@0"
I20260812 07:03:09.339591 22909 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:03:09.342754 22929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:09.343955 22909 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 07:03:09.344149 22909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "f729c2efda9346f2a36c4ba13c5c56f9"
format_stamp: "Formatted at 2026-08-12 07:03:09 on dist-test-slave-lgr5"
server_key: "0c9cf76243513f4bf01a0b8271c91ec4"
server_key_iv: "98347f0d23fca03f1b71916735cd2a72"
server_key_version: "encryptionkey@0"
I20260812 07:03:09.344322 22909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:09.368260 22909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:09.371719 22909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:09.372067 22909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:09.388608 22909 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:45769
I20260812 07:03:09.388639 23048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:45769 every 8 connection(s)
I20260812 07:03:09.389829 22909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:03:09.393258 23050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:09.396873 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 22909
I20260812 07:03:09.397111 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:03:09.397966 12035 external_mini_cluster.cc:1428] Setting key 26b6dd48697b1561da3021a85be334ee
I20260812 07:03:09.406018 23050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9: Bootstrap starting.
I20260812 07:03:09.409520 23050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:09.410760 23050 log.cc:824] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9: Log is configured to *not* fsync() on all Append() calls
Aug 12 07:03:09 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM
I20260812 07:03:09.413628 23050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9: No bootstrap required, opened a new log
I20260812 07:03:09.420859 23056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:09.399959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34494 (local address 127.11.192.254:45769)
0812 07:03:09.400709 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:09.400720 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:09.400758 (+ 38us) server_negotiation.cc:407] Connection header received
0812 07:03:09.402223 (+ 1465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:09.402244 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:09.402621 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:09.402928 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:09.404684 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:09.405948 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:09.407642 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:09.408344 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:09.408481 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:09.412878 (+ 4397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:09.412889 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:09.412906 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:09.412953 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:09.416769 (+ 3816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:09.417383 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:09.417391 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:09.417399 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:09.417613 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:09.417910 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:09.417915 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:09.417919 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:09.418569 (+ 650us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:09.418890 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:09.419176 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 07:03:09.419587 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":365,"threads_started":1}
I20260812 07:03:09.423695 23050 raft_consensus.cc:348] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } }
I20260812 07:03:09.423995 23050 raft_consensus.cc:374] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:09.424070 23050 raft_consensus.cc:729] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f729c2efda9346f2a36c4ba13c5c56f9, State: Initialized, Role: FOLLOWER
I20260812 07:03:09.424571 23050 consensus_queue.cc:260] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } }
I20260812 07:03:09.424770 23050 raft_consensus.cc:388] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:03:09.424863 23050 raft_consensus.cc:482] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:03:09.424975 23050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:09.426891 23050 raft_consensus.cc:504] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } }
I20260812 07:03:09.427333 23050 leader_election.cc:302] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f729c2efda9346f2a36c4ba13c5c56f9; no voters:
I20260812 07:03:09.428090 23050 leader_election.cc:288] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:03:09.428207 23059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:03:09.428821 23059 raft_consensus.cc:686] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [term 1 LEADER]: Becoming Leader. State: Replica: f729c2efda9346f2a36c4ba13c5c56f9, State: Running, Role: LEADER
I20260812 07:03:09.429327 23059 consensus_queue.cc:237] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } }
I20260812 07:03:09.429893 23050 sys_catalog.cc:564] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:03:09.431907 23060 sys_catalog.cc:455] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } }
I20260812 07:03:09.432297 23060 sys_catalog.cc:458] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:09.432288 23061 sys_catalog.cc:455] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f729c2efda9346f2a36c4ba13c5c56f9. Latest consensus state: current_term: 1 leader_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } }
I20260812 07:03:09.432628 23061 sys_catalog.cc:458] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:09.434195 23069 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:03:09.439535 23069 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:03:09.448268 23069 catalog_manager.cc:1347] Generated new cluster ID: 13611160523f4246a5374c1ee4313dbb
I20260812 07:03:09.448385 23069 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:03:09.457823 23069 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:03:09.459463 23069 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:03:09.472869 23069 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9: Generated new TSK 0
I20260812 07:03:09.473784 23069 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.193.keytab.
Entry for principal oryx/127.11.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.193.keytab.
I20260812 07:03:09.529366 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:45769
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.193.keytab
--principal=oryx/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:36469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:09.847342 23096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:09.847597 23096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:09.847725 23096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:09.854715 23096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:09.854810 23096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:09.854959 23096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:03:09.864269 23096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:36469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.193.keytab
--principal=oryx/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:45769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:09.865401 23096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:09.866657 23096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:09.879050 23105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:09.879094 23108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:09.879127 23110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:09.880069 23096 server_base.cc:1034] running on GCE node
I20260812 07:03:09.880631 23096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:09.881419 23096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:09.882624 23096 hybrid_clock.cc:648] HybridClock initialized: now 1786518189882572 us; error 44 us; skew 500 ppm
I20260812 07:03:09.882912 23096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:09 dist-test-slave-lgr5 krb5kdc[22890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518189, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:09.885907 23096 init.cc:377] Logged in from keytab as oryx/127.11.192.193@KRBTEST.COM (short username oryx)
I20260812 07:03:09.887205 23096 webserver.cc:460] Webserver started at http://127.11.192.193:34251/ using document root <none> and password file <none>
I20260812 07:03:09.887804 23096 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:09.887902 23096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:09.888198 23096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:09.890885 23096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "c58ec4c6df7d4bed947f921c427777c7"
format_stamp: "Formatted at 2026-08-12 07:03:09 on dist-test-slave-lgr5"
server_key: "938c8c092a765bba85d11035b6e19897"
server_key_iv: "0b71fd580f04246843dd5b2c12844300"
server_key_version: "encryptionkey@0"
I20260812 07:03:09.891626 23096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "c58ec4c6df7d4bed947f921c427777c7"
format_stamp: "Formatted at 2026-08-12 07:03:09 on dist-test-slave-lgr5"
server_key: "938c8c092a765bba85d11035b6e19897"
server_key_iv: "0b71fd580f04246843dd5b2c12844300"
server_key_version: "encryptionkey@0"
I20260812 07:03:09.896396 23096 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260812 07:03:09.900153 23131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:09.901413 23096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:03:09.901571 23096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "c58ec4c6df7d4bed947f921c427777c7"
format_stamp: "Formatted at 2026-08-12 07:03:09 on dist-test-slave-lgr5"
server_key: "938c8c092a765bba85d11035b6e19897"
server_key_iv: "0b71fd580f04246843dd5b2c12844300"
server_key_version: "encryptionkey@0"
I20260812 07:03:09.901738 23096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:09.922199 23096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:09.925801 23096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:09.926049 23096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:09.926875 23096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:09.928145 23096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:09.928231 23096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:09.928349 23096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:09.928401 23096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:09.957041 23096 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:42031
I20260812 07:03:09.957094 23364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:42031 every 8 connection(s)
I20260812 07:03:09.958176 23096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:03:09.968096 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 23096
I20260812 07:03:09.968307 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:03:09.969002 12035 external_mini_cluster.cc:1428] Setting key b9a6a623005c7190affb3a1f9ccbb2bd
Aug 12 07:03:09 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518189, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM
I20260812 07:03:09.978736 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:09.960367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:34075 (local address 127.11.192.254:45769)
0812 07:03:09.960749 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:09.960754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:09.961563 (+ 809us) server_negotiation.cc:407] Connection header received
0812 07:03:09.963316 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:09.963322 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:09.963398 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:09.963504 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:09.965019 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:09.965739 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:09.967224 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:09.967619 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:09.967733 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:09.971105 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:09.971111 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:09.971115 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:09.971155 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:09.974408 (+ 3253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:09.975806 (+ 1398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:09.975810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:09.975813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:09.975977 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:09.976681 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:09.976686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:09.976689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:09.977065 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:09.977222 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:09.978137 (+ 915us) server_negotiation.cc:299] Negotiation successful
0812 07:03:09.978375 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":197,"threads_started":1}
I20260812 07:03:09.979770 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:09.960759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.11.192.193:34075)
0812 07:03:09.961425 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:09.961460 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:09.963063 (+ 1603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:09.963734 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:09.963741 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:09.964302 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:09.964830 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:09.964852 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:09.965888 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:09.965892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:09.967084 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:09.967097 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:09.967855 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:09.967864 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:09.968102 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:09.968753 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:09.968774 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:09.970882 (+ 2108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:09.974571 (+ 3689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:09.974580 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:09.974595 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:09.975505 (+ 910us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:09.976179 (+ 674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:09.976186 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:09.976190 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:09.976537 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:09.977299 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:09.977307 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:09.977715 (+ 408us) client_negotiation.cc:769] Sending connection context
0812 07:03:09.978017 (+ 302us) client_negotiation.cc:240] Negotiation successful
0812 07:03:09.978504 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":254,"threads_started":1}
I20260812 07:03:09.981554 23367 heartbeater.cc:347] Connected to a master server at 127.11.192.254:45769
I20260812 07:03:09.981876 23367 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:09.982733 23367 heartbeater.cc:511] Master 127.11.192.254:45769 requested a full tablet report, sending...
I20260812 07:03:09.984918 22968 ts_manager.cc:194] Registered new tserver with Master: c58ec4c6df7d4bed947f921c427777c7 (127.11.192.193:42031)
I20260812 07:03:09.986770 22968 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:34075
WARNING: no policy specified for oryx/127.11.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.194.keytab.
Entry for principal oryx/127.11.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.194.keytab.
I20260812 07:03:10.020783 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:45769
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.194.keytab
--principal=oryx/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:36469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:10.341096 23381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:10.341351 23381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:10.341454 23381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:10.348691 23381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:10.348793 23381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:10.348917 23381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:03:10.358199 23381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:36469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.194.keytab
--principal=oryx/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:45769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:10.359257 23381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:10.360538 23381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:10.373169 23390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:10.373090 23393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:10.373267 23400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:10.373974 23381 server_base.cc:1034] running on GCE node
I20260812 07:03:10.374636 23381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:10.375522 23381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:10.376734 23381 hybrid_clock.cc:648] HybridClock initialized: now 1786518190376676 us; error 39 us; skew 500 ppm
I20260812 07:03:10.377097 23381 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:10 dist-test-slave-lgr5 krb5kdc[22890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518190, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:10.383677 23381 init.cc:377] Logged in from keytab as oryx/127.11.192.194@KRBTEST.COM (short username oryx)
I20260812 07:03:10.385012 23381 webserver.cc:460] Webserver started at http://127.11.192.194:41049/ using document root <none> and password file <none>
I20260812 07:03:10.385634 23381 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:10.385742 23381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:10.386029 23381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:10.388877 23381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "fcfbae397ccb4992985a1147955cba30"
format_stamp: "Formatted at 2026-08-12 07:03:10 on dist-test-slave-lgr5"
server_key: "c5a22a985623759ee2abd98926b5e586"
server_key_iv: "47388ab1e5f910a55f5dced0ea9c6bfd"
server_key_version: "encryptionkey@0"
I20260812 07:03:10.389618 23381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "fcfbae397ccb4992985a1147955cba30"
format_stamp: "Formatted at 2026-08-12 07:03:10 on dist-test-slave-lgr5"
server_key: "c5a22a985623759ee2abd98926b5e586"
server_key_iv: "47388ab1e5f910a55f5dced0ea9c6bfd"
server_key_version: "encryptionkey@0"
I20260812 07:03:10.394474 23381 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:03:10.397794 23415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:10.399101 23381 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:03:10.399256 23381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "fcfbae397ccb4992985a1147955cba30"
format_stamp: "Formatted at 2026-08-12 07:03:10 on dist-test-slave-lgr5"
server_key: "c5a22a985623759ee2abd98926b5e586"
server_key_iv: "47388ab1e5f910a55f5dced0ea9c6bfd"
server_key_version: "encryptionkey@0"
I20260812 07:03:10.399473 23381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:10.434434 23381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:10.438067 23381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:10.438304 23381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:10.439144 23381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:10.440415 23381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:10.440485 23381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:10.440599 23381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:10.440646 23381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:10.470350 23381 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:34145
I20260812 07:03:10.470407 23627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:34145 every 8 connection(s)
I20260812 07:03:10.471474 23381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:03:10.478541 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 23381
I20260812 07:03:10.478742 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:03:10.479287 12035 external_mini_cluster.cc:1428] Setting key ef8800b27c095fb4c881f3a30c9fcfac
Aug 12 07:03:10 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518190, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM
I20260812 07:03:10.490574 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:10.473913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:34317 (local address 127.11.192.254:45769)
0812 07:03:10.474101 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:10.474107 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:10.475209 (+ 1102us) server_negotiation.cc:407] Connection header received
0812 07:03:10.476956 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:10.476960 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:10.477027 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:10.477110 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:10.478438 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:10.479503 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:10.481144 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:10.481593 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:10.481692 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:10.485311 (+ 3619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:10.485316 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:10.485319 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:10.485350 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:10.487692 (+ 2342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:10.488446 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:10.488450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:10.488452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:10.488580 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:10.488989 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:10.488992 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:10.488995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:10.489237 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:10.489380 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:10.490047 (+ 667us) server_negotiation.cc:299] Negotiation successful
0812 07:03:10.490269 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:03:10.491379 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:10.474385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.11.192.194:34317)
0812 07:03:10.475054 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:10.475099 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:10.476730 (+ 1631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:10.477290 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:10.477298 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:10.477749 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:10.478279 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:10.478295 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:10.479655 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:10.479659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:10.481015 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:10.481028 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:10.481824 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:10.481831 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:10.482106 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:10.482711 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:10.482746 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:10.485105 (+ 2359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:10.487857 (+ 2752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:10.487866 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:10.487880 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:10.488323 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:10.488685 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:10.488690 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:10.488692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:10.488865 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:10.489362 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:10.489369 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:10.489664 (+ 295us) client_negotiation.cc:769] Sending connection context
0812 07:03:10.489888 (+ 224us) client_negotiation.cc:240] Negotiation successful
0812 07:03:10.490357 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":257,"threads_started":1}
I20260812 07:03:10.492758 23629 heartbeater.cc:347] Connected to a master server at 127.11.192.254:45769
I20260812 07:03:10.493016 23629 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:10.493686 23629 heartbeater.cc:511] Master 127.11.192.254:45769 requested a full tablet report, sending...
I20260812 07:03:10.495100 22968 ts_manager.cc:194] Registered new tserver with Master: fcfbae397ccb4992985a1147955cba30 (127.11.192.194:34145)
I20260812 07:03:10.495870 22968 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:34317
WARNING: no policy specified for oryx/127.11.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.195.keytab.
Entry for principal oryx/127.11.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.195.keytab.
I20260812 07:03:10.526824 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:45769
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.195.keytab
--principal=oryx/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:36469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:10.835253 23645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:10.835505 23645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:10.835602 23645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:10.842463 23645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:10.842558 23645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:10.842671 23645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:03:10.851753 23645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:36469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.195.keytab
--principal=oryx/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:45769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:10.852924 23645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:10.854070 23645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:10.866391 23660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:10.866513 23656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:10.866556 23657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:10.867443 23645 server_base.cc:1034] running on GCE node
I20260812 07:03:10.867973 23645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:10.868834 23645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:10.870020 23645 hybrid_clock.cc:648] HybridClock initialized: now 1786518190869965 us; error 44 us; skew 500 ppm
I20260812 07:03:10.870285 23645 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:10 dist-test-slave-lgr5 krb5kdc[22890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518190, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:10.873148 23645 init.cc:377] Logged in from keytab as oryx/127.11.192.195@KRBTEST.COM (short username oryx)
I20260812 07:03:10.874445 23645 webserver.cc:460] Webserver started at http://127.11.192.195:32995/ using document root <none> and password file <none>
I20260812 07:03:10.875083 23645 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:10.875198 23645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:10.875483 23645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:10.878288 23645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "476ffa1a46d444158f741ef6f66efada"
format_stamp: "Formatted at 2026-08-12 07:03:10 on dist-test-slave-lgr5"
server_key: "201d6f6163bf55e312d5080f5b0dc52e"
server_key_iv: "8d2ada8b316ce552d59540d47a7a11d9"
server_key_version: "encryptionkey@0"
I20260812 07:03:10.879021 23645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "476ffa1a46d444158f741ef6f66efada"
format_stamp: "Formatted at 2026-08-12 07:03:10 on dist-test-slave-lgr5"
server_key: "201d6f6163bf55e312d5080f5b0dc52e"
server_key_iv: "8d2ada8b316ce552d59540d47a7a11d9"
server_key_version: "encryptionkey@0"
I20260812 07:03:10.883781 23645 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 07:03:10.886943 23672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:10.888150 23645 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:03:10.888298 23645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "476ffa1a46d444158f741ef6f66efada"
format_stamp: "Formatted at 2026-08-12 07:03:10 on dist-test-slave-lgr5"
server_key: "201d6f6163bf55e312d5080f5b0dc52e"
server_key_iv: "8d2ada8b316ce552d59540d47a7a11d9"
server_key_version: "encryptionkey@0"
I20260812 07:03:10.888481 23645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:10.903494 23645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:10.907577 23645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:10.907826 23645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:10.908738 23645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:10.909988 23645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:10.910065 23645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:10.910178 23645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:10.910229 23645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:10.947432 23645 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:41935
I20260812 07:03:10.947499 23902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:41935 every 8 connection(s)
I20260812 07:03:10.948750 23645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:03:10.954177 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 23645
I20260812 07:03:10.954387 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:03:10.954967 12035 external_mini_cluster.cc:1428] Setting key 0a37454b49957fc938ff22257127ef04
Aug 12 07:03:10 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518190, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM
I20260812 07:03:10.970712 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:10.951457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:53843 (local address 127.11.192.254:45769)
0812 07:03:10.951677 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:10.951683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:10.952731 (+ 1048us) server_negotiation.cc:407] Connection header received
0812 07:03:10.954562 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:10.954567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:10.954661 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:10.954752 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:10.956467 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:10.957703 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:10.960040 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:10.960690 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:10.960817 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:10.964847 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:10.964852 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:10.964855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:10.964886 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:10.967314 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:10.968219 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:10.968225 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:10.968228 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:10.968382 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:10.968873 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:10.968878 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:10.968882 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:10.969212 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:10.969356 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:10.970125 (+ 769us) server_negotiation.cc:299] Negotiation successful
0812 07:03:10.970354 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 07:03:10.971644 23910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:10.951796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.11.192.195:53843)
0812 07:03:10.952568 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:10.952619 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:10.954248 (+ 1629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:10.954997 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:10.955005 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:10.955536 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:10.956276 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:10.956305 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:10.957900 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:10.957906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:10.959863 (+ 1957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:10.959886 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:10.960911 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:10.960918 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:10.961204 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:10.961917 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:10.961946 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:10.964642 (+ 2696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:10.967453 (+ 2811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:10.967462 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:10.967478 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:10.968067 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:10.968498 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:10.968503 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:10.968507 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:10.968746 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:10.969332 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:10.969340 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:10.969713 (+ 373us) client_negotiation.cc:769] Sending connection context
0812 07:03:10.969991 (+ 278us) client_negotiation.cc:240] Negotiation successful
0812 07:03:10.970467 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":332,"threads_started":1}
I20260812 07:03:10.973062 23903 heartbeater.cc:347] Connected to a master server at 127.11.192.254:45769
I20260812 07:03:10.973394 23903 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:10.976523 23903 heartbeater.cc:511] Master 127.11.192.254:45769 requested a full tablet report, sending...
I20260812 07:03:10.978133 22968 ts_manager.cc:194] Registered new tserver with Master: 476ffa1a46d444158f741ef6f66efada (127.11.192.195:41935)
I20260812 07:03:10.979184 22968 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:53843
I20260812 07:03:10.982653 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:03:10.989614 23367 heartbeater.cc:503] Master 127.11.192.254:45769 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.11.192.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.192.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.192.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.253.keytab.
Entry for principal oryx/127.11.192.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.253.keytab.
I20260812 07:03:11.043499 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.253:41945
--webserver_interface=127.11.192.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.253.keytab
--principal=oryx/127.11.192.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:36469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:45769,127.11.192.253:41945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:11.373085 23915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:11.373358 23915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:11.373482 23915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:11.380386 23915 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:03:11.380486 23915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:11.380554 23915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:11.380627 23915 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:03:11.380685 23915 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:03:11.391215 23915 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:36469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:45769,127.11.192.253:41945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.253.keytab
--principal=oryx/127.11.192.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.253:41945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.192.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
W20260812 07:03:11.391543 23915 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.11.192.254:45769,127.11.192.253:41945'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 07:03:11.392714 23915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:11.394258 23915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:11.405999 23924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:11.406114 23926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:11.406138 23928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:11.412575 23915 server_base.cc:1034] running on GCE node
I20260812 07:03:11.413241 23915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:11.414399 23915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:11.415599 23915 hybrid_clock.cc:648] HybridClock initialized: now 1786518191415526 us; error 48 us; skew 500 ppm
I20260812 07:03:11.415901 23915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:11 dist-test-slave-lgr5 krb5kdc[22890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518191, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:11.418834 23915 init.cc:377] Logged in from keytab as oryx/127.11.192.253@KRBTEST.COM (short username oryx)
I20260812 07:03:11.420212 23915 webserver.cc:460] Webserver started at http://127.11.192.253:40643/ using document root <none> and password file <none>
I20260812 07:03:11.420833 23915 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:11.420914 23915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:11.421149 23915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:11.423796 23915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/data/instance:
uuid: "8d82aaf8d7ef451dabbed9397999081c"
format_stamp: "Formatted at 2026-08-12 07:03:11 on dist-test-slave-lgr5"
server_key: "2bc5071dd5e5444f528835ba8a7ab8bf"
server_key_iv: "955f0c3c61e5fa221613c881a0c88c8f"
server_key_version: "encryptionkey@0"
I20260812 07:03:11.424515 23915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/wal/instance:
uuid: "8d82aaf8d7ef451dabbed9397999081c"
format_stamp: "Formatted at 2026-08-12 07:03:11 on dist-test-slave-lgr5"
server_key: "2bc5071dd5e5444f528835ba8a7ab8bf"
server_key_iv: "955f0c3c61e5fa221613c881a0c88c8f"
server_key_version: "encryptionkey@0"
I20260812 07:03:11.429039 23915 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:03:11.432114 23948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:11.433279 23915 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260812 07:03:11.433425 23915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/wal
uuid: "8d82aaf8d7ef451dabbed9397999081c"
format_stamp: "Formatted at 2026-08-12 07:03:11 on dist-test-slave-lgr5"
server_key: "2bc5071dd5e5444f528835ba8a7ab8bf"
server_key_iv: "955f0c3c61e5fa221613c881a0c88c8f"
server_key_version: "encryptionkey@0"
I20260812 07:03:11.433600 23915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:11.445811 23915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:11.449543 23915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:11.449754 23915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:11.465124 23915 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.253:41945
I20260812 07:03:11.465153 24039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.253:41945 every 8 connection(s)
I20260812 07:03:11.466307 23915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/data/info.pb
I20260812 07:03:11.469628 24040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:11.473858 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 23915
I20260812 07:03:11.476038 24040 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:03:11 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518191, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.253@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM
Aug 12 07:03:11 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.192.253@KRBTEST.COM
I20260812 07:03:11.495610 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:11.478545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44192 (local address 127.11.192.254:45769)
0812 07:03:11.478986 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:11.478992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:11.479611 (+ 619us) server_negotiation.cc:407] Connection header received
0812 07:03:11.479682 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:11.479687 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:11.479763 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:11.479878 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:11.481449 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.482593 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.484770 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.485423 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.485559 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:11.489401 (+ 3842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:11.489408 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:11.489413 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:11.489458 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:11.492236 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:11.493044 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:11.493050 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:11.493053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:11.493182 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:11.493642 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:11.493646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:11.493648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:11.494061 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:11.494225 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:11.494995 (+ 770us) server_negotiation.cc:299] Negotiation successful
0812 07:03:11.495248 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":224,"threads_started":1}
I20260812 07:03:11.496522 24047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:11.478608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.0.0.1:44192)
0812 07:03:11.479296 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:11.479327 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:11.479564 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:11.479969 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:11.479976 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:11.480772 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:11.481280 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:11.481300 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.482714 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.482717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:11.484630 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:11.484647 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.485675 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.485683 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:11.485884 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:11.486552 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:11.486585 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:11.489209 (+ 2624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:11.492354 (+ 3145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:11.492364 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:11.492371 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:11.492926 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:11.493286 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:11.493290 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:11.493293 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:11.493516 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:11.494194 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:11.494202 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:11.494592 (+ 390us) client_negotiation.cc:769] Sending connection context
0812 07:03:11.494813 (+ 221us) client_negotiation.cc:240] Negotiation successful
0812 07:03:11.495233 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"spinlock_wait_cycles":24192,"thread_start_us":384,"threads_started":1}
I20260812 07:03:11.496564 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:11.475921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46652 (local address 127.11.192.253:41945)
0812 07:03:11.476722 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:11.476733 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:11.476769 (+ 36us) server_negotiation.cc:407] Connection header received
0812 07:03:11.478571 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:11.478596 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:11.479061 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:11.479436 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:11.481224 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.482870 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.484698 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.485446 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.485606 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:11.490270 (+ 4664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:11.490280 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:11.490292 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:11.490325 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:11.493276 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:11.493966 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:11.493975 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:11.493978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:11.494137 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:11.494519 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:11.494522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:11.494524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:11.494893 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:11.495216 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:11.495555 (+ 339us) server_negotiation.cc:299] Negotiation successful
0812 07:03:11.496010 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":396,"threads_started":1}
I20260812 07:03:11.498170 24040 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:03:11.498181 23629 heartbeater.cc:503] Master 127.11.192.254:45769 was elected leader, sending a full tablet report...
Aug 12 07:03:11 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518191, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.253@KRBTEST.COM for oryx/127.11.192.253@KRBTEST.COM
I20260812 07:03:11.509445 24047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:11.498870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41945 (local address 127.0.0.1:46662)
0812 07:03:11.499021 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:11.499039 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:11.499125 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:11.499502 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:11.499504 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:11.499867 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:11.500039 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:11.500051 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.500941 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.500947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:11.501764 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:11.501775 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.502376 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.502381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:11.502541 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:11.503030 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:11.503050 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:11.504858 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:11.507329 (+ 2471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:11.507334 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:11.507338 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:11.507701 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:11.508012 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:11.508016 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:11.508019 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:11.508178 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:11.508633 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:11.508638 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:11.508760 (+ 122us) client_negotiation.cc:769] Sending connection context
0812 07:03:11.508860 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 07:03:11.509119 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 07:03:11.509583 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:11.499082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46662 (local address 127.11.192.253:41945)
0812 07:03:11.499239 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:11.499245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:11.499263 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:03:11.499320 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:11.499325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:11.499386 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:11.499493 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:11.500169 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.500810 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.501888 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.502254 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.502355 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:11.505014 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:11.505018 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:11.505020 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:11.505046 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:11.507213 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:11.507803 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:11.507807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:11.507809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:11.507909 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:11.508272 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:11.508277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:11.508280 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:11.508528 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:11.508649 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:11.508981 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 07:03:11.509265 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":90,"server-negotiator.queue_time_us":50}
I20260812 07:03:11.513747 24040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c: Bootstrap starting.
I20260812 07:03:11.516690 24040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:11.517634 24040 log.cc:824] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c: Log is configured to *not* fsync() on all Append() calls
I20260812 07:03:11.519912 24040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c: No bootstrap required, opened a new log
I20260812 07:03:11.526685 24040 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } }
I20260812 07:03:11.526978 24040 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:11.527055 24040 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d82aaf8d7ef451dabbed9397999081c, State: Initialized, Role: FOLLOWER
I20260812 07:03:11.527514 24040 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } }
I20260812 07:03:11.528275 24052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } } }
I20260812 07:03:11.528555 24052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:03:11.529124 24040 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: configured and running, proceeding with master startup.
W20260812 07:03:11.537181 24068 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:03:11.537304 24068 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:03:11.539544 23915 master_runner.cc:428] Detected that this master 8d82aaf8d7ef451dabbed9397999081c is joining an existing cluster
I20260812 07:03:11.539644 23915 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.192.253:41945
I20260812 07:03:11.558257 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:11.544066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44196 (local address 127.11.192.254:45769)
0812 07:03:11.544269 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:11.544275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:11.544404 (+ 129us) server_negotiation.cc:407] Connection header received
0812 07:03:11.544646 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:11.544652 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:11.544740 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:11.544812 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:11.546379 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.547130 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.548727 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.549196 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.549304 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:11.551436 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:11.551443 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:11.551449 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:11.551500 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:11.555220 (+ 3720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:11.556074 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:11.556103 (+ 29us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:11.556106 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:11.556248 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:11.556688 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:11.556692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:11.556695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:11.557018 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:11.557128 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:11.557735 (+ 607us) server_negotiation.cc:299] Negotiation successful
0812 07:03:11.557964 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:03:11.559278 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:11.543604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46678 (local address 127.11.192.253:41945)
0812 07:03:11.543797 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:11.543803 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:11.545118 (+ 1315us) server_negotiation.cc:407] Connection header received
0812 07:03:11.545274 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:11.545279 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:11.545335 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:11.545416 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:11.546510 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.547638 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.549623 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.550361 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.550454 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:11.552803 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:11.552809 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:11.552813 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:11.552849 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:11.556320 (+ 3471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:11.557123 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:11.557128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:11.557131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:11.557279 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:11.557699 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:11.557704 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:11.557707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:11.558062 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:11.558202 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:11.558664 (+ 462us) server_negotiation.cc:299] Negotiation successful
0812 07:03:11.558887 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:03:11.559816 24079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:11.543632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.0.0.1:44196)
0812 07:03:11.544307 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:11.544325 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:11.544500 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:11.545041 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:11.545046 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:11.545724 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:11.546205 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:11.546220 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.547290 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.547295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:11.548599 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:11.548611 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.549387 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.549394 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:11.549565 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:11.550149 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:11.550169 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:11.551228 (+ 1059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:11.555368 (+ 4140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:11.555372 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:11.555375 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:11.555947 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:11.556391 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:11.556395 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:11.556398 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:11.556566 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:11.557226 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:11.557234 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:11.557430 (+ 196us) client_negotiation.cc:769] Sending connection context
0812 07:03:11.558707 (+ 1277us) client_negotiation.cc:240] Negotiation successful
0812 07:03:11.559050 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"mutex_wait_us":96,"thread_start_us":643,"threads_started":1}
I20260812 07:03:11.560288 24080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:11.543717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41945 (local address 127.0.0.1:46678)
0812 07:03:11.545024 (+ 1307us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:11.545038 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:11.545163 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:11.545450 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:11.545458 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:11.546021 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:11.546369 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:11.546380 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.547843 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.547848 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:11.549473 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:11.549488 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.550765 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.550773 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:11.550962 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:11.551547 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:11.551569 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:11.552639 (+ 1070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:11.556464 (+ 3825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:11.556468 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:11.556472 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:11.556998 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:11.557415 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:11.557419 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:11.557422 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:11.557585 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:11.558268 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:11.558272 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:11.558413 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 07:03:11.559524 (+ 1111us) client_negotiation.cc:240] Negotiation successful
0812 07:03:11.559792 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1200,"mutex_wait_us":44,"thread_start_us":999,"threads_started":1}
Aug 12 07:03:11 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518189, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.254@KRBTEST.COM for oryx/127.11.192.253@KRBTEST.COM
WARNING: no policy specified for oryx/127.11.192.252@KRBTEST.COM; defaulting to no policy
I20260812 07:03:11.581640 24084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:11.566242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41945 (local address 127.0.0.1:46684)
0812 07:03:11.566865 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:11.566903 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:11.567057 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:11.567442 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:11.567447 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:11.568023 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:11.568272 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:11.568282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.569516 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.569520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:11.570511 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:11.570523 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.571502 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.571509 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:11.571712 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:11.572384 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:11.572407 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:11.575095 (+ 2688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:11.578970 (+ 3875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:11.578983 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:11.578998 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:11.579533 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:11.579975 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:11.579980 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:11.579984 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:11.580175 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:11.580792 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:11.580798 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:11.580942 (+ 144us) client_negotiation.cc:769] Sending connection context
0812 07:03:11.581095 (+ 153us) client_negotiation.cc:240] Negotiation successful
0812 07:03:11.581348 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":252,"threads_started":1}
I20260812 07:03:11.582032 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:11.566300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46684 (local address 127.11.192.253:41945)
0812 07:03:11.566466 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:11.566472 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:11.566999 (+ 527us) server_negotiation.cc:407] Connection header received
0812 07:03:11.567226 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:11.567231 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:11.567317 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:11.567420 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:11.568413 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.569375 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.570649 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.571289 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.571435 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:11.575281 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:11.575288 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:11.575293 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:11.575327 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:11.578826 (+ 3499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:11.579671 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:11.579677 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:11.579681 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:11.579848 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:11.580301 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:11.580306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:11.580309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:11.580643 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:11.580787 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:11.581234 (+ 447us) server_negotiation.cc:299] Negotiation successful
0812 07:03:11.581677 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.192.252@KRBTEST.COM" created.
I20260812 07:03:11.583750 22968 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.192.253:41945: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c"
member_type: NON_VOTER
last_known_addr {
host: "127.11.192.253"
port: 41945
}
attrs {
promote: true
}
}
dest_uuid: "f729c2efda9346f2a36c4ba13c5c56f9"
cas_config_opid_index: -1
I20260812 07:03:11.584703 22968 consensus_queue.cc:237] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: NON_VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: true } }
I20260812 07:03:11.598156 24002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:11.602106 24002 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 1 FOLLOWER]: Refusing update from remote peer f729c2efda9346f2a36c4ba13c5c56f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 07:03:11.609546 24087 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: NON_VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:03:11.609426 24096 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 8d82aaf8d7ef451dabbed9397999081c (127.11.192.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: NON_VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: true } } }
I20260812 07:03:11.613085 24096 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.192.253:41945
I20260812 07:03:11.613794 24096 sys_catalog.cc:455] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: NON_VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: true } } }
I20260812 07:03:11.613988 24096 sys_catalog.cc:458] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:11.614084 23915 master.cc:537] Master@127.11.192.253:41945 shutting down...
I20260812 07:03:11.627820 24002 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 8d82aaf8d7ef451dabbed9397999081c (127.11.192.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: NON_VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: true } } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.192.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.252.keytab.
Entry for principal oryx/127.11.192.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.252.keytab.
I20260812 07:03:11.633548 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.252:36667
--webserver_interface=127.11.192.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.252.keytab
--principal=oryx/127.11.192.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:36469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:45769,127.11.192.253:41945,127.11.192.252:36667
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:03:11.651273 24113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: NON_VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: true } } }
I20260812 07:03:11.651643 24113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: This master's current role is: LEARNER
I20260812 07:03:11.651975 24052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f729c2efda9346f2a36c4ba13c5c56f9. Latest consensus state: current_term: 1 leader_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: NON_VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: true } } }
I20260812 07:03:11.652251 24052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: This master's current role is: LEARNER
I20260812 07:03:11.651952 24103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: NON_VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: true } } }
I20260812 07:03:11.652722 24103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: This master's current role is: LEARNER
I20260812 07:03:11.654743 23915 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 1 LEARNER]: Raft consensus shutting down.
I20260812 07:03:11.655233 23915 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 1 LEARNER]: Raft consensus is shut down!
I20260812 07:03:11.655404 23915 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c: stopping tablet replica
I20260812 07:03:11.673722 23915 master.cc:552] Master@127.11.192.253:41945 shutdown complete.
I20260812 07:03:11.673898 23915 master_runner.cc:305] Clearing existing system tablet
I20260812 07:03:11.674364 23915 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:11.674465 23915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:11.680785 23915 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.004s
I20260812 07:03:11.683420 24115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:11.683790 23915 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:11.684128 23915 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 07:03:11.684226 23915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/wal
uuid: "8d82aaf8d7ef451dabbed9397999081c"
format_stamp: "Formatted at 2026-08-12 07:03:11 on dist-test-slave-lgr5"
server_key: "2bc5071dd5e5444f528835ba8a7ab8bf"
server_key_iv: "955f0c3c61e5fa221613c881a0c88c8f"
server_key_version: "encryptionkey@0"
I20260812 07:03:11.688753 23915 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:03:11.695573 23915 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 07:03:11.695762 23915 log.cc:1196] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 07:03:11.696355 23915 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c: Deleting consensus metadata
I20260812 07:03:11.696645 23915 master_runner.cc:315] Copying system tablet from 127.11.192.254:45769
I20260812 07:03:11.701920 23915 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c: tablet copy: Beginning tablet copy session from remote peer at address 127.11.192.254:45769
I20260812 07:03:11.717651 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:11.702793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44200 (local address 127.11.192.254:45769)
0812 07:03:11.703024 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:11.703030 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:11.703672 (+ 642us) server_negotiation.cc:407] Connection header received
0812 07:03:11.703872 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:11.703878 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:11.703978 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:11.704104 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:11.705126 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.706284 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.708118 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.708782 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.708905 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:11.711237 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:11.711244 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:11.711249 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:11.711288 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:11.714680 (+ 3392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:11.715466 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:11.715472 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:11.715476 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:11.715640 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:11.716073 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:11.716104 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:11.716108 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:11.716443 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:11.716605 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:11.717067 (+ 462us) server_negotiation.cc:299] Negotiation successful
0812 07:03:11.717283 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:03:11.717694 24126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:11.702770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.0.0.1:44200)
0812 07:03:11.703466 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:11.703555 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:11.703724 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:11.704164 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:11.704169 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:11.704673 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:11.704956 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:11.704979 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.706477 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.706483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:11.707936 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:11.707953 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:11.708955 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:11.708961 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:11.709153 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:11.709877 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:11.709900 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:11.711008 (+ 1108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:11.714819 (+ 3811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:11.714824 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:11.714828 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:11.715342 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:11.715773 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:11.715779 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:11.715783 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:11.715953 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:11.716588 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:11.716593 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:11.716784 (+ 191us) client_negotiation.cc:769] Sending connection context
0812 07:03:11.716962 (+ 178us) client_negotiation.cc:240] Negotiation successful
0812 07:03:11.717267 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":415,"threads_started":1}
I20260812 07:03:11.718873 23018 tablet_copy_service.cc:140] P f729c2efda9346f2a36c4ba13c5c56f9: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8d82aaf8d7ef451dabbed9397999081c ({username='oryx', principal='oryx/127.11.192.253@KRBTEST.COM'} at 127.0.0.1:44200)
I20260812 07:03:11.719154 23018 tablet_copy_service.cc:161] P f729c2efda9346f2a36c4ba13c5c56f9: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8d82aaf8d7ef451dabbed9397999081c at {username='oryx', principal='oryx/127.11.192.253@KRBTEST.COM'} at 127.0.0.1:44200: session id = 8d82aaf8d7ef451dabbed9397999081c-00000000000000000000000000000000
I20260812 07:03:11.723639 23018 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 07:03:11.725724 23915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 07:03:11.732432 23915 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c: tablet copy: Starting download of 0 data blocks...
I20260812 07:03:11.732728 23915 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c: tablet copy: Starting download of 1 WAL segments...
I20260812 07:03:11.735271 23915 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 07:03:11.739562 23018 tablet_copy_service.cc:342] P f729c2efda9346f2a36c4ba13c5c56f9: Request end of tablet copy session 8d82aaf8d7ef451dabbed9397999081c-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.192.253@KRBTEST.COM'} at 127.0.0.1:44200
I20260812 07:03:11.739775 23018 tablet_copy_service.cc:433] P f729c2efda9346f2a36c4ba13c5c56f9: ending tablet copy session 8d82aaf8d7ef451dabbed9397999081c-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8d82aaf8d7ef451dabbed9397999081c
I20260812 07:03:11.750798 23915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:11.753268 24131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:11.755439 24134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:11.765188 24138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:11.765800 23915 server_base.cc:1034] running on GCE node
I20260812 07:03:11.766148 23915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:11.766776 23915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:11.768025 23915 hybrid_clock.cc:648] HybridClock initialized: now 1786518191767936 us; error 35 us; skew 500 ppm
I20260812 07:03:11.768359 23915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:11 dist-test-slave-lgr5 krb5kdc[22890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518191, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:11.770601 23915 init.cc:377] Logged in from keytab as oryx/127.11.192.253@KRBTEST.COM (short username oryx)
I20260812 07:03:11.772998 23915 webserver.cc:460] Webserver started at http://127.11.192.253:38013/ using document root <none> and password file <none>
I20260812 07:03:11.773365 23915 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:11.773453 23915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:11.776618 23915 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 07:03:11.778514 24147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:11.778947 23915 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260812 07:03:11.779129 23915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/wal
uuid: "8d82aaf8d7ef451dabbed9397999081c"
format_stamp: "Formatted at 2026-08-12 07:03:11 on dist-test-slave-lgr5"
server_key: "2bc5071dd5e5444f528835ba8a7ab8bf"
server_key_iv: "955f0c3c61e5fa221613c881a0c88c8f"
server_key_version: "encryptionkey@0"
I20260812 07:03:11.779374 23915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:11.787724 23915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:11.791932 23915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:11.820546 23915 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.253:41945
I20260812 07:03:11.820576 24230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.253:41945 every 8 connection(s)
I20260812 07:03:11.821569 23915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-1/data/info.pb
I20260812 07:03:11.826989 24233 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:03:11.830683 24233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c: Bootstrap starting.
I20260812 07:03:11.845081 24233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:03:11.845577 24233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c: Bootstrap complete.
I20260812 07:03:11.846583 24233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: NON_VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: true } }
I20260812 07:03:11.846844 24233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8d82aaf8d7ef451dabbed9397999081c, State: Initialized, Role: LEARNER
I20260812 07:03:11.847105 24233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: NON_VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: true } }
I20260812 07:03:11.847980 24236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: NON_VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: true } } }
I20260812 07:03:11.848359 24233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:03:11.848578 24236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: This master's current role is: LEARNER
I20260812 07:03:11.858770 24256 catalog_manager.cc:1259] Loaded cluster ID: 13611160523f4246a5374c1ee4313dbb
I20260812 07:03:11.858867 24256 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c: loading cluster ID for follower catalog manager: success
I20260812 07:03:11.862396 24256 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c: acquiring CA information for follower catalog manager: success
I20260812 07:03:11.864871 24256 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 07:03:11.982018 23903 heartbeater.cc:503] Master 127.11.192.254:45769 was elected leader, sending a full tablet report...
I20260812 07:03:12.002240 24088 sys_catalog.cc:455] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: NON_VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: true } } }
I20260812 07:03:12.002652 24088 sys_catalog.cc:458] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:12.008483 24084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.002472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41945 (local address 127.0.0.1:46688)
0812 07:03:12.002739 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:12.002759 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:12.002881 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:12.003715 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:12.003719 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:12.003755 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:12.003922 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.003931 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.005246 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.005251 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.006589 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.006603 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.007706 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.007713 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.007930 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.007980 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 07:03:12.008066 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 07:03:12.008219 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260812 07:03:12.008589 24258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.002651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46688 (local address 127.11.192.253:41945)
0812 07:03:12.003396 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:12.003401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:12.003417 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:03:12.003475 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:12.003479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:12.003548 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:12.003655 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:12.004064 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.005101 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.006731 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.007543 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.007721 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.007910 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:12.008126 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 07:03:12.008255 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":460,"threads_started":1}
I20260812 07:03:12.009622 24236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f729c2efda9346f2a36c4ba13c5c56f9. Latest consensus state: current_term: 1 leader_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: NON_VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: true } } }
I20260812 07:03:12.009912 24236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: This master's current role is: LEARNER
I20260812 07:03:12.010521 24096 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9: attempting to promote NON_VOTER 8d82aaf8d7ef451dabbed9397999081c to VOTER
I20260812 07:03:12.011267 24096 consensus_queue.cc:237] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } }
I20260812 07:03:12.012826 24188 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 1 LEARNER]: Refusing update from remote peer f729c2efda9346f2a36c4ba13c5c56f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 07:03:12.013370 24088 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 07:03:12.016131 24096 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 8d82aaf8d7ef451dabbed9397999081c (127.11.192.253) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } }
I20260812 07:03:12.016814 24188 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 8d82aaf8d7ef451dabbed9397999081c (127.11.192.253) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } }
W20260812 07:03:12.017989 24110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:12.018313 24110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:12.018457 24110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 07:03:12.018708 24096 sys_catalog.cc:455] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } }
I20260812 07:03:12.018959 24096 sys_catalog.cc:458] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:12.018937 24087 sys_catalog.cc:455] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } }
I20260812 07:03:12.019213 24087 sys_catalog.cc:458] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:12.020975 24236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } }
I20260812 07:03:12.021178 24236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: This master's current role is: FOLLOWER
W20260812 07:03:12.026577 24110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:03:12.026679 24110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:12.026753 24110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:12.026829 24110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:03:12.026890 24110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:03:12.036465 24110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:36469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:45769,127.11.192.253:41945,127.11.192.252:36667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.252.keytab
--principal=oryx/127.11.192.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.252:36667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.192.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:12.037664 24110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:12.038942 24110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:12.050166 24269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:12.050233 24275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:12.050256 24271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:12.053347 24110 server_base.cc:1034] running on GCE node
I20260812 07:03:12.054082 24110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:12.055353 24110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:12.056550 24110 hybrid_clock.cc:648] HybridClock initialized: now 1786518192056497 us; error 40 us; skew 500 ppm
I20260812 07:03:12.056856 24110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:12 dist-test-slave-lgr5 krb5kdc[22890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518192, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:12.059767 24110 init.cc:377] Logged in from keytab as oryx/127.11.192.252@KRBTEST.COM (short username oryx)
I20260812 07:03:12.061146 24110 webserver.cc:460] Webserver started at http://127.11.192.252:34661/ using document root <none> and password file <none>
I20260812 07:03:12.061764 24110 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:12.061861 24110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:12.062111 24110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:12.064868 24110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/data/instance:
uuid: "998954c32f6b45cba1277aacf2dc5196"
format_stamp: "Formatted at 2026-08-12 07:03:12 on dist-test-slave-lgr5"
server_key: "34a07bce76801445fdbce66e98c0b510"
server_key_iv: "435d8b4b356b816de85bc8fb95367745"
server_key_version: "encryptionkey@0"
I20260812 07:03:12.065560 24110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/wal/instance:
uuid: "998954c32f6b45cba1277aacf2dc5196"
format_stamp: "Formatted at 2026-08-12 07:03:12 on dist-test-slave-lgr5"
server_key: "34a07bce76801445fdbce66e98c0b510"
server_key_iv: "435d8b4b356b816de85bc8fb95367745"
server_key_version: "encryptionkey@0"
I20260812 07:03:12.070480 24110 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:03:12.073648 24295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:12.074822 24110 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:03:12.074960 24110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/wal
uuid: "998954c32f6b45cba1277aacf2dc5196"
format_stamp: "Formatted at 2026-08-12 07:03:12 on dist-test-slave-lgr5"
server_key: "34a07bce76801445fdbce66e98c0b510"
server_key_iv: "435d8b4b356b816de85bc8fb95367745"
server_key_version: "encryptionkey@0"
I20260812 07:03:12.075150 24110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:12.091516 24110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:12.095345 24110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:12.095554 24110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:12.112638 24110 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.252:36667
I20260812 07:03:12.112680 24377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.252:36667 every 8 connection(s)
I20260812 07:03:12.113813 24110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/data/info.pb
I20260812 07:03:12.117195 24378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:12.121384 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 24110
I20260812 07:03:12.124305 24378 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:03:12 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518192, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.252@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM
Aug 12 07:03:12 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.192.252@KRBTEST.COM
I20260812 07:03:12.144240 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.128223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44210 (local address 127.11.192.254:45769)
0812 07:03:12.128405 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:12.128410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:12.129125 (+ 715us) server_negotiation.cc:407] Connection header received
0812 07:03:12.129310 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:12.129314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:12.129381 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:12.129471 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:12.130940 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.132191 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.134087 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.134526 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.134631 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.137731 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:12.137735 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:12.137738 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:12.137770 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:12.140921 (+ 3151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.141910 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.141915 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.141917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.142061 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.142449 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.142454 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.142458 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.142816 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:12.142930 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:12.143694 (+ 764us) server_negotiation.cc:299] Negotiation successful
0812 07:03:12.143889 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 07:03:12.145596 24385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.127938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.0.0.1:44210)
0812 07:03:12.128974 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:12.129024 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:12.129168 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:12.129610 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:12.129616 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:12.130399 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:12.130783 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.130793 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.132366 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.132370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.133962 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.133972 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.134729 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.134736 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.134872 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.135474 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:12.135510 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:12.137554 (+ 2044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:12.141086 (+ 3532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:12.141096 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:12.141110 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:12.141795 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:12.142171 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:12.142175 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:12.142179 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:12.142344 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:12.142933 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:12.142940 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:12.143277 (+ 337us) client_negotiation.cc:769] Sending connection context
0812 07:03:12.143613 (+ 336us) client_negotiation.cc:240] Negotiation successful
0812 07:03:12.144072 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"mutex_wait_us":33,"thread_start_us":622,"threads_started":1}
I20260812 07:03:12.145615 24384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.123340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38938 (local address 127.11.192.252:36667)
0812 07:03:12.124131 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:12.124142 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:12.124182 (+ 40us) server_negotiation.cc:407] Connection header received
0812 07:03:12.126201 (+ 2019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:12.126222 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:12.126596 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:12.126911 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:12.128489 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.130486 (+ 1997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.132472 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.133479 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.133647 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.138338 (+ 4691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:12.138346 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:12.138356 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:12.138382 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:12.142062 (+ 3680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.142800 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.142806 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.142808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.142941 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.143281 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.143285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.143288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.143712 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:12.143994 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:12.144326 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 07:03:12.144559 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":574,"thread_start_us":355,"threads_started":1}
I20260812 07:03:12.147094 24378 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:03:12 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518192, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.252@KRBTEST.COM for oryx/127.11.192.253@KRBTEST.COM
I20260812 07:03:12.158241 24385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.147710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41945 (local address 127.0.0.1:46692)
0812 07:03:12.147858 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:12.147875 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:12.148006 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:12.148382 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:12.148386 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:12.148715 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:12.148863 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.148871 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.149853 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.149857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.150757 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.150769 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.151376 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.151380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.151539 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.151963 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:12.151980 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:12.153831 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:12.156474 (+ 2643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:12.156478 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:12.156481 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:12.156816 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:12.157111 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:12.157113 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:12.157115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:12.157230 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:12.157617 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:12.157619 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:12.157723 (+ 104us) client_negotiation.cc:769] Sending connection context
0812 07:03:12.157820 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 07:03:12.158006 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 07:03:12.158278 24258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.147862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46692 (local address 127.11.192.253:41945)
0812 07:03:12.148114 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:12.148121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:12.148140 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:03:12.148198 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:12.148201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:12.148260 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:12.148338 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:12.149002 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.149740 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.150861 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.151266 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.151354 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.154001 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:12.154006 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:12.154009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:12.154041 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:12.156365 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.156896 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.156899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.156901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.157034 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.157301 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.157303 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.157305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.157537 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:12.157640 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:12.157885 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 07:03:12.158059 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:03:12.159114 24378 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.252" port: 36667 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:03:12 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518192, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.252@KRBTEST.COM for oryx/127.11.192.252@KRBTEST.COM
I20260812 07:03:12.169770 24385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.159636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:36667 (local address 127.0.0.1:38944)
0812 07:03:12.159772 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:12.159786 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:12.159855 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:12.160068 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:12.160071 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:12.160390 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:12.160534 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.160542 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.161370 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.161374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.162162 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.162172 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.162757 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.162761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.162903 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.163377 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:12.163393 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:12.165120 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:12.167823 (+ 2703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:12.167826 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:12.167829 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:12.168230 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:12.168529 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:12.168532 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:12.168534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:12.168681 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:12.169108 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:12.169111 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:12.169216 (+ 105us) client_negotiation.cc:769] Sending connection context
0812 07:03:12.169331 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 07:03:12.169503 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 07:03:12.169819 24384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.159699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38944 (local address 127.11.192.252:36667)
0812 07:03:12.159841 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:12.159844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:12.159858 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:03:12.159921 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:12.159923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:12.159970 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:12.160043 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:12.160642 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.161271 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.162268 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.162647 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.162736 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.165282 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:12.165286 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:12.165289 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:12.165316 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:12.167705 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.168319 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.168322 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.168324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.168440 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.168772 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.168775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.168776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.169019 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:12.169119 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:12.169407 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 07:03:12.169609 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":60}
I20260812 07:03:12.174185 24378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196: Bootstrap starting.
I20260812 07:03:12.177166 24378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:12.178133 24378 log.cc:824] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196: Log is configured to *not* fsync() on all Append() calls
I20260812 07:03:12.180718 24378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196: No bootstrap required, opened a new log
I20260812 07:03:12.187489 24378 raft_consensus.cc:348] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 36667 } }
I20260812 07:03:12.187769 24378 raft_consensus.cc:374] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:12.187879 24378 raft_consensus.cc:729] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998954c32f6b45cba1277aacf2dc5196, State: Initialized, Role: FOLLOWER
I20260812 07:03:12.188448 24378 consensus_queue.cc:260] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 36667 } }
I20260812 07:03:12.189239 24389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 36667 } } }
I20260812 07:03:12.189522 24389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:03:12.190114 24378 sys_catalog.cc:564] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 07:03:12.198298 24405 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:03:12.198417 24405 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:03:12.200052 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 23096
I20260812 07:03:12.202778 24110 master_runner.cc:428] Detected that this master 998954c32f6b45cba1277aacf2dc5196 is joining an existing cluster
I20260812 07:03:12.202900 24110 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.192.252:36667
Aug 12 07:03:12 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.192.193@KRBTEST.COM
I20260812 07:03:12.219448 24407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.201240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58232 (local address 127.11.192.193:42031)
0812 07:03:12.202569 (+ 1329us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:12.202577 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:12.202603 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:03:12.202679 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:12.202687 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:12.202905 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:12.203124 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:12.204622 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.205861 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.207084 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.207652 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.207764 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.211850 (+ 4086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:12.211861 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:12.211872 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:12.211907 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:12.215160 (+ 3253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.215972 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.215979 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.215982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.216179 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.216658 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.216664 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.216667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.217118 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:12.218569 (+ 1451us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:12.218734 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 07:03:12.219012 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":324,"threads_started":1}
I20260812 07:03:12.221040 23358 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:12.230333 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.214992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44214 (local address 127.11.192.254:45769)
0812 07:03:12.215177 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:12.215183 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:12.215293 (+ 110us) server_negotiation.cc:407] Connection header received
0812 07:03:12.215562 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:12.215568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:12.215647 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:12.215715 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:12.217288 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.218461 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.220531 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.221149 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.221244 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.223596 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:12.223603 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:12.223607 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:12.223647 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:12.227024 (+ 3377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.227819 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.227825 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.227828 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.227992 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.228536 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.228541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.228544 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.228884 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:12.229017 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:12.229723 (+ 706us) server_negotiation.cc:299] Negotiation successful
0812 07:03:12.229984 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 07:03:12.231338 24416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.214553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.0.0.1:44214)
0812 07:03:12.215181 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:12.215200 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:12.215396 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:12.215927 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:12.215931 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:12.216743 (+ 812us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:12.217114 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.217127 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.218663 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.218668 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.220350 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.220366 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.221543 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.221549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.221725 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.222310 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:12.222332 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:12.223380 (+ 1048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:12.227202 (+ 3822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:12.227207 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:12.227211 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:12.227680 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:12.228175 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:12.228180 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:12.228184 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:12.228378 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:12.229122 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:12.229127 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:12.229364 (+ 237us) client_negotiation.cc:769] Sending connection context
0812 07:03:12.230600 (+ 1236us) client_negotiation.cc:240] Negotiation successful
0812 07:03:12.230900 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":625,"threads_started":1}
I20260812 07:03:12.246114 24384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.227219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38956 (local address 127.11.192.252:36667)
0812 07:03:12.227388 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:12.227393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:12.231631 (+ 4238us) server_negotiation.cc:407] Connection header received
0812 07:03:12.231820 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:12.231825 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:12.231885 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:12.231941 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:12.232951 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.234066 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.235987 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.236759 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.236853 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.239236 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:12.239242 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:12.239247 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:12.239285 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:12.242992 (+ 3707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.243785 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.243790 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.243793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.243964 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.244434 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.244443 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.244447 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.244828 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:12.244999 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:12.245532 (+ 533us) server_negotiation.cc:299] Negotiation successful
0812 07:03:12.245753 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:03:12.247017 24416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.227501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:36667 (local address 127.0.0.1:38956)
0812 07:03:12.231557 (+ 4056us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:12.231574 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:12.231697 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:12.232125 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:12.232129 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:12.232602 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:12.232809 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.232820 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.234275 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.234280 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.235842 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.235858 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.237106 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.237115 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.237276 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.237873 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:12.237895 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:12.238965 (+ 1070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:12.243148 (+ 4183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:12.243152 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:12.243156 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:12.243663 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:12.244130 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:12.244134 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:12.244138 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:12.244314 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:12.245090 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:12.245094 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:12.245244 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 07:03:12.246329 (+ 1085us) client_negotiation.cc:240] Negotiation successful
0812 07:03:12.246637 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3950,"threads_started":1}
Aug 12 07:03:12 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518189, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.254@KRBTEST.COM for oryx/127.11.192.252@KRBTEST.COM
I20260812 07:03:12.259684 24258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.215896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46700 (local address 127.11.192.253:41945)
0812 07:03:12.216108 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:12.216115 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:12.226693 (+ 10578us) server_negotiation.cc:407] Connection header received
0812 07:03:12.227976 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:12.227982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:12.228060 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:12.228147 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:12.230812 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.231852 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.248264 (+ 16412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.248992 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.249099 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.252960 (+ 3861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:12.252966 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:12.252971 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:12.253011 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:12.256390 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.257311 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.257316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.257319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.257488 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.257991 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.257996 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.258000 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.258353 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:12.258505 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:12.259096 (+ 591us) server_negotiation.cc:299] Negotiation successful
0812 07:03:12.259325 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:03:12.260643 24417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.216536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41945 (local address 127.0.0.1:46700)
0812 07:03:12.226594 (+ 10058us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:12.226613 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:12.227708 (+ 1095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:12.228373 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:12.228378 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:12.229031 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:12.229386 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.229403 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.244907 (+ 15504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.244919 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.246612 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.246623 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.250611 (+ 3988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.250622 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.250825 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.251581 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:12.251602 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:12.252745 (+ 1143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:12.256578 (+ 3833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:12.256583 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:12.256587 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:12.257167 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:12.257639 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:12.257644 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:12.257648 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:12.257851 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:12.258608 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:12.258612 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:12.258794 (+ 182us) client_negotiation.cc:769] Sending connection context
0812 07:03:12.259860 (+ 1066us) client_negotiation.cc:240] Negotiation successful
0812 07:03:12.260229 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9926,"thread_start_us":10166,"threads_started":1}
I20260812 07:03:12.260800 24084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.237299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:36667 (local address 127.0.0.1:38964)
0812 07:03:12.237445 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:12.237461 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:12.237601 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:12.245353 (+ 7752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:12.245358 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:12.245797 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:12.245962 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.245973 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.248699 (+ 2726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.248704 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.250044 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.250060 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.250921 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.250926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.251086 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.251542 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:12.251562 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:12.254117 (+ 2555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:12.258127 (+ 4010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:12.258132 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:12.258137 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:12.258606 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:12.259038 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:12.259043 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:12.259047 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:12.259190 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:12.259810 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:12.259815 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:12.259970 (+ 155us) client_negotiation.cc:769] Sending connection context
0812 07:03:12.260135 (+ 165us) client_negotiation.cc:240] Negotiation successful
0812 07:03:12.260420 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 07:03:12.263095 24424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.237720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38964 (local address 127.11.192.252:36667)
0812 07:03:12.244950 (+ 7230us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:12.244955 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:12.244980 (+ 25us) server_negotiation.cc:407] Connection header received
0812 07:03:12.245075 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:12.245080 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:12.245170 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:12.246114 (+ 944us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:12.246611 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.248269 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.250221 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.250787 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.250893 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.254347 (+ 3454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:12.254353 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:12.254357 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:12.254390 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:12.257988 (+ 3598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.258760 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.258765 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.258769 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.258920 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.259317 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.259322 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.259326 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.259672 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:12.262392 (+ 2720us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:12.262515 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 07:03:12.262711 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7105,"thread_start_us":6973,"threads_started":1}
I20260812 07:03:12.264328 22968 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.192.252:36667: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "998954c32f6b45cba1277aacf2dc5196"
member_type: NON_VOTER
last_known_addr {
host: "127.11.192.252"
port: 36667
}
attrs {
promote: true
}
}
dest_uuid: "f729c2efda9346f2a36c4ba13c5c56f9"
cas_config_opid_index: 6
I20260812 07:03:12.265071 22968 consensus_queue.cc:237] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: NON_VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: true } }
I20260812 07:03:12.267112 24188 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 1 FOLLOWER]: Refusing update from remote peer f729c2efda9346f2a36c4ba13c5c56f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 07:03:12.267304 24338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:12.267658 24087 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 07:03:12.270170 24088 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 998954c32f6b45cba1277aacf2dc5196 (127.11.192.252) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: NON_VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: true } } }
I20260812 07:03:12.270704 24188 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 998954c32f6b45cba1277aacf2dc5196 (127.11.192.252) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: NON_VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: true } } }
I20260812 07:03:12.274349 24236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: NON_VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: true } } }
I20260812 07:03:12.274609 24236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:03:12.275661 24338 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [term 1 FOLLOWER]: Refusing update from remote peer f729c2efda9346f2a36c4ba13c5c56f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 07:03:12.276283 24088 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.192.252:36667
I20260812 07:03:12.276489 24087 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: NON_VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:03:12.276822 24110 master.cc:537] Master@127.11.192.252:36667 shutting down...
W20260812 07:03:12.279342 22938 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 07:03:12.286060 22938 consensus_peers.cc:487] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 -> Peer 998954c32f6b45cba1277aacf2dc5196 (127.11.192.252:36667): Couldn't send request to peer 998954c32f6b45cba1277aacf2dc5196. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 07:03:12.287935 24110 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:03:12.288405 24110 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:03:12.288681 24110 tablet_replica.cc:330] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196: stopping tablet replica
I20260812 07:03:12.291539 24096 sys_catalog.cc:455] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: NON_VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: true } } }
I20260812 07:03:12.291857 24096 sys_catalog.cc:458] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:12.293335 24088 sys_catalog.cc:455] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: NON_VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: true } } }
I20260812 07:03:12.293694 24088 sys_catalog.cc:458] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:12.296180 24110 master.cc:552] Master@127.11.192.252:36667 shutdown complete.
I20260812 07:03:12.296257 24110 master_runner.cc:305] Clearing existing system tablet
I20260812 07:03:12.296638 24110 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:12.296727 24110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:12.299584 24110 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:03:12.301216 24437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:12.301529 24110 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:12.301908 24110 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 07:03:12.302052 24110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/wal
uuid: "998954c32f6b45cba1277aacf2dc5196"
format_stamp: "Formatted at 2026-08-12 07:03:12 on dist-test-slave-lgr5"
server_key: "34a07bce76801445fdbce66e98c0b510"
server_key_iv: "435d8b4b356b816de85bc8fb95367745"
server_key_version: "encryptionkey@0"
I20260812 07:03:12.307185 24110 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:03:12.313967 24110 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 07:03:12.314178 24110 log.cc:1196] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 07:03:12.314678 24110 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196: Deleting consensus metadata
I20260812 07:03:12.314960 24110 master_runner.cc:315] Copying system tablet from 127.11.192.254:45769
I20260812 07:03:12.317553 24110 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196: tablet copy: Beginning tablet copy session from remote peer at address 127.11.192.254:45769
I20260812 07:03:12.333074 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.318362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44218 (local address 127.11.192.254:45769)
0812 07:03:12.318570 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:12.318576 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:12.319462 (+ 886us) server_negotiation.cc:407] Connection header received
0812 07:03:12.319620 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:12.319625 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:12.319719 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:12.319826 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:12.320866 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.322009 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.323804 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.324458 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.324596 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.326776 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:12.326783 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:12.326788 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:12.326829 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:12.330280 (+ 3451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.331022 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.331027 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.331031 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.331183 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.331565 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.331569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.331573 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.331923 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:12.332075 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:12.332514 (+ 439us) server_negotiation.cc:299] Negotiation successful
0812 07:03:12.332741 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260812 07:03:12.333117 24444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.318307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.0.0.1:44218)
0812 07:03:12.319344 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:12.319362 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:12.319503 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:12.319863 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:12.319868 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:12.320437 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:12.320708 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.320721 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.322142 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.322148 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.323648 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.323664 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.324629 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.324636 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.324804 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.325522 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:12.325544 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:12.326571 (+ 1027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:12.330402 (+ 3831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:12.330407 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:12.330411 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:12.330895 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:12.331284 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:12.331288 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:12.331292 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:12.331463 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:12.332041 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:12.332046 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:12.332251 (+ 205us) client_negotiation.cc:769] Sending connection context
0812 07:03:12.332388 (+ 137us) client_negotiation.cc:240] Negotiation successful
0812 07:03:12.332674 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"thread_start_us":783,"threads_started":1}
I20260812 07:03:12.334112 23018 tablet_copy_service.cc:140] P f729c2efda9346f2a36c4ba13c5c56f9: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 998954c32f6b45cba1277aacf2dc5196 ({username='oryx', principal='oryx/127.11.192.252@KRBTEST.COM'} at 127.0.0.1:44218)
I20260812 07:03:12.334281 23018 tablet_copy_service.cc:161] P f729c2efda9346f2a36c4ba13c5c56f9: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 998954c32f6b45cba1277aacf2dc5196 at {username='oryx', principal='oryx/127.11.192.252@KRBTEST.COM'} at 127.0.0.1:44218: session id = 998954c32f6b45cba1277aacf2dc5196-00000000000000000000000000000000
I20260812 07:03:12.337894 23018 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 07:03:12.339622 24110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 07:03:12.346184 24110 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196: tablet copy: Starting download of 0 data blocks...
I20260812 07:03:12.346447 24110 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196: tablet copy: Starting download of 1 WAL segments...
I20260812 07:03:12.348933 24110 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 07:03:12.353094 23018 tablet_copy_service.cc:342] P f729c2efda9346f2a36c4ba13c5c56f9: Request end of tablet copy session 998954c32f6b45cba1277aacf2dc5196-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.192.252@KRBTEST.COM'} at 127.0.0.1:44218
I20260812 07:03:12.353222 23018 tablet_copy_service.cc:433] P f729c2efda9346f2a36c4ba13c5c56f9: ending tablet copy session 998954c32f6b45cba1277aacf2dc5196-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 998954c32f6b45cba1277aacf2dc5196
I20260812 07:03:12.363536 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 23096
I20260812 07:03:12.366796 24110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:12.377758 24445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:12.384033 24447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:12.388203 24110 server_base.cc:1034] running on GCE node
W20260812 07:03:12.388198 24449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:12.388722 24110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:12.389277 24110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:12.390451 24110 hybrid_clock.cc:648] HybridClock initialized: now 1786518192390419 us; error 39 us; skew 500 ppm
I20260812 07:03:12.390645 24110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:12 dist-test-slave-lgr5 krb5kdc[22890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518192, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:12.392578 24110 init.cc:377] Logged in from keytab as oryx/127.11.192.252@KRBTEST.COM (short username oryx)
I20260812 07:03:12.399291 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:42031
--local_ip_for_outbound_sockets=127.11.192.193
--tserver_master_addrs=127.11.192.254:45769
--webserver_port=34251
--webserver_interface=127.11.192.193
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.193.keytab
--principal=oryx/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:36469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.192.254:45769,127.11.192.253:41945
--tserver_master_addrs=127.11.192.254:45769,127.11.192.253:41945,127.11.192.252:36667 with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:03:12.409226 24110 webserver.cc:460] Webserver started at http://127.11.192.252:38709/ using document root <none> and password file <none>
I20260812 07:03:12.409608 24110 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:12.409704 24110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:12.412724 24110 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 07:03:12.414634 24465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:12.415084 24110 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260812 07:03:12.415256 24110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/wal
uuid: "998954c32f6b45cba1277aacf2dc5196"
format_stamp: "Formatted at 2026-08-12 07:03:12 on dist-test-slave-lgr5"
server_key: "34a07bce76801445fdbce66e98c0b510"
server_key_iv: "435d8b4b356b816de85bc8fb95367745"
server_key_version: "encryptionkey@0"
I20260812 07:03:12.415508 24110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:12.448947 24110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:12.452922 24110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:12.480216 24110 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.252:36667
I20260812 07:03:12.480257 24571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.252:36667 every 8 connection(s)
I20260812 07:03:12.481279 24110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/master-2/data/info.pb
I20260812 07:03:12.486899 24572 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:03:12.491156 24572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196: Bootstrap starting.
I20260812 07:03:12.505678 24572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:03:12.506132 24572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196: Bootstrap complete.
I20260812 07:03:12.506984 24572 raft_consensus.cc:348] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: NON_VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: true } }
I20260812 07:03:12.507208 24572 raft_consensus.cc:729] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 998954c32f6b45cba1277aacf2dc5196, State: Initialized, Role: LEARNER
I20260812 07:03:12.507432 24572 consensus_queue.cc:260] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: NON_VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: true } }
I20260812 07:03:12.508490 24572 sys_catalog.cc:564] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:03:12.508291 24576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: NON_VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: true } } }
I20260812 07:03:12.508848 24576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [sys.catalog]: This master's current role is: LEARNER
I20260812 07:03:12.519738 24595 catalog_manager.cc:1259] Loaded cluster ID: 13611160523f4246a5374c1ee4313dbb
I20260812 07:03:12.519814 24595 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196: loading cluster ID for follower catalog manager: success
I20260812 07:03:12.522302 24595 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196: acquiring CA information for follower catalog manager: success
I20260812 07:03:12.524021 24595 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 07:03:12.720525 24598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.714007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38980 (local address 127.11.192.252:36667)
0812 07:03:12.714530 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:12.714535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:12.714552 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:03:12.714612 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:12.714616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:12.714688 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:12.714786 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:12.715224 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.716307 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.718604 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.719491 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.719665 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.719847 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:12.720166 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 07:03:12.720271 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":287,"threads_started":1}
I20260812 07:03:12.720515 24084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.713813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:36667 (local address 127.0.0.1:38980)
0812 07:03:12.714015 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:12.714037 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:12.714179 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:12.714849 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:12.714855 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:12.714886 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:12.715082 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.715095 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.716441 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.716447 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.718425 (+ 1978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.718444 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.719695 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.719701 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.719935 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.719975 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 07:03:12.720073 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 07:03:12.720233 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 07:03:12.721581 24521 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.7] Dedup: 1.7->[]
I20260812 07:03:12.722115 24576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f729c2efda9346f2a36c4ba13c5c56f9. Latest consensus state: current_term: 1 leader_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: NON_VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: true } } }
I20260812 07:03:12.722390 24576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [sys.catalog]: This master's current role is: LEARNER
W20260812 07:03:12.757205 24461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:12.757479 24461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:12.757620 24461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:12.764688 24461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:12.764794 24461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:12.764943 24461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:03:12.774271 24461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:36469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.193.keytab
--principal=oryx/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:42031
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=34251
--tserver_master_addrs=127.11.192.254:45769,127.11.192.253:41945,127.11.192.252:36667
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:12.775607 24461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:12.776882 24461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:12.789318 24603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:12.789281 24604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:12.789373 24609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:12.790418 24461 server_base.cc:1034] running on GCE node
I20260812 07:03:12.790983 24461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:12.791973 24461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:12.793169 24461 hybrid_clock.cc:648] HybridClock initialized: now 1786518192793118 us; error 59 us; skew 500 ppm
I20260812 07:03:12.793457 24461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:12 dist-test-slave-lgr5 krb5kdc[22890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518192, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:12.796628 24461 init.cc:377] Logged in from keytab as oryx/127.11.192.193@KRBTEST.COM (short username oryx)
I20260812 07:03:12.797896 24461 webserver.cc:460] Webserver started at http://127.11.192.193:34251/ using document root <none> and password file <none>
I20260812 07:03:12.798516 24461 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:12.798646 24461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:12.803781 24461 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:03:12.806317 24624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:12.807536 24461 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 07:03:12.807684 24461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "c58ec4c6df7d4bed947f921c427777c7"
format_stamp: "Formatted at 2026-08-12 07:03:09 on dist-test-slave-lgr5"
server_key: "938c8c092a765bba85d11035b6e19897"
server_key_iv: "0b71fd580f04246843dd5b2c12844300"
server_key_version: "encryptionkey@0"
I20260812 07:03:12.808687 24461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:12.834904 24461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:12.838593 24461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:12.838843 24461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:12.839737 24461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:12.841099 24461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:12.841194 24461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:12.841296 24461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:12.841364 24461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:12.871757 24461 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:42031
I20260812 07:03:12.871821 24825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:42031 every 8 connection(s)
I20260812 07:03:12.873101 24461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:03:12.883493 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 24461
I20260812 07:03:12.883785 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 23381
Aug 12 07:03:12 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518192, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.252@KRBTEST.COM
Aug 12 07:03:12 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518192, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM
Aug 12 07:03:12 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518192, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.253@KRBTEST.COM
I20260812 07:03:12.911263 24598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.875792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:37299 (local address 127.11.192.252:36667)
0812 07:03:12.875981 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:12.875987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:12.884932 (+ 8945us) server_negotiation.cc:407] Connection header received
0812 07:03:12.888356 (+ 3424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:12.888363 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:12.888455 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:12.888537 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:12.892562 (+ 4025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.893755 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.897139 (+ 3384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.897871 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.897985 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.903278 (+ 5293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:12.903284 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:12.903289 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:12.903331 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:12.906596 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.907967 (+ 1371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.907981 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.907985 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.908172 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.908815 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.908820 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.908823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.909452 (+ 629us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:12.909591 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:12.910680 (+ 1089us) server_negotiation.cc:299] Negotiation successful
0812 07:03:12.910922 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 07:03:12.911604 24836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.875700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:40661 (local address 127.11.192.254:45769)
0812 07:03:12.879104 (+ 3404us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:12.879108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:12.885491 (+ 6383us) server_negotiation.cc:407] Connection header received
0812 07:03:12.891084 (+ 5593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:12.891090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:12.891179 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:12.891262 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:12.896331 (+ 5069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.897497 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.899351 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.899910 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.900031 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.904054 (+ 4023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:12.904060 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:12.904065 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:12.904128 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:12.907627 (+ 3499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.908475 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.908480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.908483 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.908651 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.909173 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.909179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.909182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.909585 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:12.909714 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:12.910683 (+ 969us) server_negotiation.cc:299] Negotiation successful
0812 07:03:12.910920 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3299,"thread_start_us":3201,"threads_started":1}
I20260812 07:03:12.913928 24838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.876379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:36667 (local address 127.11.192.193:37299)
0812 07:03:12.884734 (+ 8355us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:12.884801 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:12.888183 (+ 3382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:12.889080 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:12.889092 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:12.889998 (+ 906us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:12.890839 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.890864 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.894020 (+ 3156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.894025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.896141 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.896158 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.898428 (+ 2270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.898436 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.898740 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.899878 (+ 1138us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:12.899927 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:12.903030 (+ 3103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:12.906749 (+ 3719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:12.906757 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:12.906773 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:12.907725 (+ 952us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:12.908323 (+ 598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:12.908327 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:12.908330 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:12.908619 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:12.909694 (+ 1075us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:12.909701 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:12.910234 (+ 533us) client_negotiation.cc:769] Sending connection context
0812 07:03:12.911551 (+ 1317us) client_negotiation.cc:240] Negotiation successful
0812 07:03:12.912506 (+ 955us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8027,"spinlock_wait_cycles":13696,"thread_start_us":11620,"threads_started":1}
Aug 12 07:03:12 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.192.194@KRBTEST.COM
I20260812 07:03:12.915364 24837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.876379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.11.192.193:40661)
0812 07:03:12.885362 (+ 8983us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:12.885379 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:12.887859 (+ 2480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:12.891587 (+ 3728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:12.891591 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:12.892056 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:12.892341 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.892352 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.897696 (+ 5344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.897702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.899191 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.899202 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.900308 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.900314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.900513 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.901129 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:12.901152 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:12.903735 (+ 2583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:12.907785 (+ 4050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:12.907790 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:12.907793 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:12.908297 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:12.908833 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:12.908837 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:12.908840 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:12.909038 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:12.909794 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:12.909799 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:12.910230 (+ 431us) client_negotiation.cc:769] Sending connection context
0812 07:03:12.912157 (+ 1927us) client_negotiation.cc:240] Negotiation successful
0812 07:03:12.914893 (+ 2736us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8839,"mutex_wait_us":2465,"spinlock_wait_cycles":87296,"thread_start_us":7747,"threads_started":1}
I20260812 07:03:12.915704 24827 heartbeater.cc:347] Connected to a master server at 127.11.192.252:36667
I20260812 07:03:12.916029 24827 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:12.916251 24832 heartbeater.cc:347] Connected to a master server at 127.11.192.254:45769
I20260812 07:03:12.916391 24832 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:12.917066 24832 heartbeater.cc:511] Master 127.11.192.254:45769 requested a full tablet report, sending...
I20260812 07:03:12.917316 24827 heartbeater.cc:511] Master 127.11.192.252:36667 requested a full tablet report, sending...
I20260812 07:03:12.919194 22968 ts_manager.cc:194] Re-registered known tserver with Master: c58ec4c6df7d4bed947f921c427777c7 (127.11.192.193:42031)
I20260812 07:03:12.919816 24846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.888765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:50501 (local address 127.11.192.253:41945)
0812 07:03:12.896939 (+ 8174us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:12.896945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:12.898822 (+ 1877us) server_negotiation.cc:407] Connection header received
0812 07:03:12.899308 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:12.899314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:12.899402 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:12.899504 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:12.901275 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.903782 (+ 2507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.906321 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.906943 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.907043 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.912026 (+ 4983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:12.912034 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:12.912038 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:12.912101 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:12.915490 (+ 3389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.917569 (+ 2079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.917574 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.917578 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.917723 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.918194 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.918199 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.918202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.918560 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:12.918710 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:12.919243 (+ 533us) server_negotiation.cc:299] Negotiation successful
0812 07:03:12.919482 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8044,"thread_start_us":7919,"threads_started":1}
I20260812 07:03:12.919920 24502 ts_manager.cc:194] Registered new tserver with Master: c58ec4c6df7d4bed947f921c427777c7 (127.11.192.193:42031)
I20260812 07:03:12.920332 22968 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:40661
I20260812 07:03:12.920723 24850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.894752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41945 (local address 127.11.192.193:50501)
0812 07:03:12.898724 (+ 3972us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:12.898742 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:12.899143 (+ 401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:12.899684 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:12.899689 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:12.900379 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:12.901074 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.901090 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.904230 (+ 3140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.904235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.906145 (+ 1910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:12.906161 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.907436 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.907443 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:12.907651 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.908230 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:12.908301 (+ 71us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:12.911790 (+ 3489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:12.916894 (+ 5104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:12.916900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:12.916904 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:12.917414 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:12.917870 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:12.917874 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:12.917878 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:12.918061 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:12.918799 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:12.918805 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:12.918964 (+ 159us) client_negotiation.cc:769] Sending connection context
0812 07:03:12.920061 (+ 1097us) client_negotiation.cc:240] Negotiation successful
0812 07:03:12.920358 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3845,"thread_start_us":3712,"threads_started":1}
I20260812 07:03:12.921697 24833 heartbeater.cc:347] Connected to a master server at 127.11.192.253:41945
I20260812 07:03:12.921824 24833 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:12.922230 24845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:12.884661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56200 (local address 127.11.192.194:34145)
0812 07:03:12.896663 (+ 12002us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:12.896672 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:12.899429 (+ 2757us) server_negotiation.cc:407] Connection header received
0812 07:03:12.899626 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:12.899634 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:12.899865 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:12.900023 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:12.902260 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.903580 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.905363 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:12.906208 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:12.906327 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:12.915253 (+ 8926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:12.915266 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:12.915281 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:12.915320 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:12.918882 (+ 3562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.919726 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.919735 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.919739 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.919891 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:12.920367 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:12.920372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:12.920375 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:12.920800 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:12.921093 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:12.921681 (+ 588us) server_negotiation.cc:299] Negotiation successful
0812 07:03:12.921943 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11846,"thread_start_us":11709,"threads_started":1}
I20260812 07:03:12.922375 24833 heartbeater.cc:511] Master 127.11.192.253:41945 requested a full tablet report, sending...
I20260812 07:03:12.924016 24170 ts_manager.cc:194] Registered new tserver with Master: c58ec4c6df7d4bed947f921c427777c7 (127.11.192.193:42031)
I20260812 07:03:12.924170 23621 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:13.006299 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 23381
I20260812 07:03:13.033036 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:34145
--local_ip_for_outbound_sockets=127.11.192.194
--tserver_master_addrs=127.11.192.254:45769
--webserver_port=41049
--webserver_interface=127.11.192.194
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.194.keytab
--principal=oryx/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:36469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.192.254:45769,127.11.192.253:41945
--tserver_master_addrs=127.11.192.254:45769,127.11.192.253:41945,127.11.192.252:36667 with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:03:13.209853 24088 sys_catalog.cc:455] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: NON_VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: true } } }
I20260812 07:03:13.210247 24088 sys_catalog.cc:458] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:13.213747 24088 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9: attempting to promote NON_VOTER 998954c32f6b45cba1277aacf2dc5196 to VOTER
I20260812 07:03:13.214515 24088 consensus_queue.cc:237] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: false } }
I20260812 07:03:13.216773 24188 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 1 FOLLOWER]: Refusing update from remote peer f729c2efda9346f2a36c4ba13c5c56f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260812 07:03:13.216799 24521 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [term 1 LEARNER]: Refusing update from remote peer f729c2efda9346f2a36c4ba13c5c56f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260812 07:03:13.217437 24855 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 07:03:13.217883 24088 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 07:03:13.221331 24855 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 998954c32f6b45cba1277aacf2dc5196 (127.11.192.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: false } } }
I20260812 07:03:13.222100 24188 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 998954c32f6b45cba1277aacf2dc5196 (127.11.192.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: false } } }
I20260812 07:03:13.225031 24855 sys_catalog.cc:455] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: false } } }
I20260812 07:03:13.225282 24855 sys_catalog.cc:458] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:13.225353 24854 sys_catalog.cc:455] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: false } } }
I20260812 07:03:13.225689 24854 sys_catalog.cc:458] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:13.226136 24854 sys_catalog.cc:455] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: false } } }
I20260812 07:03:13.226411 24854 sys_catalog.cc:458] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:13.226405 24521 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 998954c32f6b45cba1277aacf2dc5196 (127.11.192.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: false } } }
I20260812 07:03:13.229612 24865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: false } } }
I20260812 07:03:13.229908 24865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:03:13.237895 24864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: false } } }
I20260812 07:03:13.238300 24864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: This master's current role is: FOLLOWER
W20260812 07:03:13.356909 24853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:13.357175 24853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:13.357280 24853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:13.364327 24853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:13.364451 24853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:13.364580 24853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:03:13.373891 24853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:36469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.194.keytab
--principal=oryx/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:34145
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=41049
--tserver_master_addrs=127.11.192.254:45769,127.11.192.253:41945,127.11.192.252:36667
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:13.375088 24853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:13.376417 24853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:13.389029 24870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:13.389338 24872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:13.389648 24875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:13.390203 24853 server_base.cc:1034] running on GCE node
I20260812 07:03:13.390823 24853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:13.391829 24853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:13.393074 24853 hybrid_clock.cc:648] HybridClock initialized: now 1786518193393016 us; error 45 us; skew 500 ppm
I20260812 07:03:13.393399 24853 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:13 dist-test-slave-lgr5 krb5kdc[22890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518193, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:13.396855 24853 init.cc:377] Logged in from keytab as oryx/127.11.192.194@KRBTEST.COM (short username oryx)
I20260812 07:03:13.398208 24853 webserver.cc:460] Webserver started at http://127.11.192.194:41049/ using document root <none> and password file <none>
I20260812 07:03:13.398823 24853 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:13.398944 24853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:13.403874 24853 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:03:13.406461 24890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:13.407652 24853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:03:13.407795 24853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "fcfbae397ccb4992985a1147955cba30"
format_stamp: "Formatted at 2026-08-12 07:03:10 on dist-test-slave-lgr5"
server_key: "c5a22a985623759ee2abd98926b5e586"
server_key_iv: "47388ab1e5f910a55f5dced0ea9c6bfd"
server_key_version: "encryptionkey@0"
I20260812 07:03:13.408766 24853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:13.430671 24853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:13.434428 24853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:13.434674 24853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:13.435559 24853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:13.436841 24853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:13.436929 24853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:13.437036 24853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:13.437105 24853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:13.466049 24853 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:34145
I20260812 07:03:13.466122 25086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:34145 every 8 connection(s)
I20260812 07:03:13.467409 24853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:03:13.477298 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 24853
I20260812 07:03:13.477581 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 23645
Aug 12 07:03:13 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518193, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.252@KRBTEST.COM
I20260812 07:03:13.496676 25098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:13.470252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:38973 (local address 127.11.192.252:36667)
0812 07:03:13.476534 (+ 6282us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:13.476540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:13.476562 (+ 22us) server_negotiation.cc:407] Connection header received
0812 07:03:13.476621 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:13.476625 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:13.476701 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:13.478987 (+ 2286us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:13.479261 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:13.480836 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:13.483437 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:13.484210 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:13.485151 (+ 941us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:13.488879 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:13.488885 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:13.488890 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:13.488943 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:13.492626 (+ 3683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:13.493529 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:13.493535 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:13.493539 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:13.493678 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:13.494192 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:13.494198 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:13.494201 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:13.494536 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:13.494714 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:13.496051 (+ 1337us) server_negotiation.cc:299] Negotiation successful
0812 07:03:13.496297 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6133,"thread_start_us":5970,"threads_started":1}
I20260812 07:03:13.497577 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:13.470884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:36667 (local address 127.11.192.194:38973)
0812 07:03:13.472760 (+ 1876us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:13.472813 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:13.475396 (+ 2583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:13.477128 (+ 1732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:13.477140 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:13.477885 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:13.478703 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:13.478731 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:13.481084 (+ 2353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:13.481090 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:13.483209 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:13.483226 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:13.484457 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:13.484465 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:13.484742 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:13.485487 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:13.485516 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:13.488610 (+ 3094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:13.492772 (+ 4162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:13.492780 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:13.492794 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:13.493371 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:13.493817 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:13.493822 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:13.493826 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:13.494060 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:13.494727 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:13.494736 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:13.495168 (+ 432us) client_negotiation.cc:769] Sending connection context
0812 07:03:13.495522 (+ 354us) client_negotiation.cc:240] Negotiation successful
0812 07:03:13.495994 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1532,"mutex_wait_us":1,"thread_start_us":1266,"threads_started":1}
Aug 12 07:03:13 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518193, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.253@KRBTEST.COM
I20260812 07:03:13.499311 25087 heartbeater.cc:347] Connected to a master server at 127.11.192.252:36667
I20260812 07:03:13.499660 25087 heartbeater.cc:464] Registering TS with master...
Aug 12 07:03:13 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518193, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM
I20260812 07:03:13.500567 25087 heartbeater.cc:511] Master 127.11.192.252:36667 requested a full tablet report, sending...
I20260812 07:03:13.502090 24502 ts_manager.cc:194] Registered new tserver with Master: fcfbae397ccb4992985a1147955cba30 (127.11.192.194:34145)
Aug 12 07:03:13 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.192.195@KRBTEST.COM
I20260812 07:03:13.506361 25103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:13.473286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41945 (local address 127.11.192.194:52519)
0812 07:03:13.485316 (+ 12030us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:13.485332 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:13.487389 (+ 2057us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:13.488161 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:13.488166 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:13.488741 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:13.489048 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:13.489059 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:13.494048 (+ 4989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:13.494051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:13.495268 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:13.495284 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:13.496401 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:13.496408 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:13.496597 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:13.497143 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:13.497195 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:13.499756 (+ 2561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:13.503852 (+ 4096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:13.503857 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:13.503861 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:13.504420 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:13.504820 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:13.504825 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:13.504829 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:13.505023 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:13.505573 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:13.505578 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:13.505727 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 07:03:13.505867 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 07:03:13.506096 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11920,"thread_start_us":11799,"threads_started":1}
I20260812 07:03:13.506624 25104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:13.473701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:52519 (local address 127.11.192.253:41945)
0812 07:03:13.476655 (+ 2954us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:13.476661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:13.485408 (+ 8747us) server_negotiation.cc:407] Connection header received
0812 07:03:13.487723 (+ 2315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:13.487729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:13.487828 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:13.487907 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:13.492841 (+ 4934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:13.493896 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:13.495411 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:13.496004 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:13.496157 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:13.499977 (+ 3820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:13.499984 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:13.499988 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:13.500040 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:13.503686 (+ 3646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:13.504544 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:13.504549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:13.504552 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:13.504695 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:13.505168 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:13.505172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:13.505174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:13.505453 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:13.505575 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:13.506022 (+ 447us) server_negotiation.cc:299] Negotiation successful
0812 07:03:13.506238 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2847,"thread_start_us":2751,"threads_started":1}
I20260812 07:03:13.507517 25094 heartbeater.cc:347] Connected to a master server at 127.11.192.253:41945
I20260812 07:03:13.507627 25094 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:13.507941 25094 heartbeater.cc:511] Master 127.11.192.253:41945 requested a full tablet report, sending...
I20260812 07:03:13.508334 25096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:13.470221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:46011 (local address 127.11.192.254:45769)
0812 07:03:13.476073 (+ 5852us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:13.476098 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:13.484332 (+ 8234us) server_negotiation.cc:407] Connection header received
0812 07:03:13.485553 (+ 1221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:13.485560 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:13.485656 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:13.485739 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:13.489697 (+ 3958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:13.490770 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:13.492592 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:13.493170 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:13.493280 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:13.501146 (+ 7866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:13.501154 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:13.501159 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:13.501212 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:13.504363 (+ 3151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:13.505102 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:13.505108 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:13.505111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:13.505270 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:13.506557 (+ 1287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:13.506563 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:13.506567 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:13.506880 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:13.507019 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:13.507650 (+ 631us) server_negotiation.cc:299] Negotiation successful
0812 07:03:13.507870 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5740,"thread_start_us":5647,"threads_started":1}
I20260812 07:03:13.509078 24170 ts_manager.cc:194] Registered new tserver with Master: fcfbae397ccb4992985a1147955cba30 (127.11.192.194:34145)
I20260812 07:03:13.509308 25102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:13.471376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.11.192.194:46011)
0812 07:03:13.484246 (+ 12870us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:13.484264 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:13.484751 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:13.485973 (+ 1222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:13.485978 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:13.486492 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:13.486805 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:13.486818 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:13.490989 (+ 4171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:13.490994 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:13.492438 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:13.492455 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:13.495501 (+ 3046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:13.495507 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:13.495652 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:13.496953 (+ 1301us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:13.496977 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:13.500561 (+ 3584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:13.504480 (+ 3919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:13.504486 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:13.504490 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:13.504982 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:13.505738 (+ 756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:13.505743 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:13.505746 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:13.505883 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:13.507167 (+ 1284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:13.507171 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:13.507319 (+ 148us) client_negotiation.cc:769] Sending connection context
0812 07:03:13.508675 (+ 1356us) client_negotiation.cc:240] Negotiation successful
0812 07:03:13.508930 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12752,"thread_start_us":5443,"threads_started":1}
I20260812 07:03:13.510023 25091 heartbeater.cc:347] Connected to a master server at 127.11.192.254:45769
I20260812 07:03:13.510138 25091 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:13.510504 25091 heartbeater.cc:511] Master 127.11.192.254:45769 requested a full tablet report, sending...
I20260812 07:03:13.511365 22968 ts_manager.cc:194] Re-registered known tserver with Master: fcfbae397ccb4992985a1147955cba30 (127.11.192.194:34145)
I20260812 07:03:13.512486 22968 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:46011
I20260812 07:03:13.513512 25110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:13.478494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39892 (local address 127.11.192.195:41935)
0812 07:03:13.492656 (+ 14162us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:13.492664 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:13.492690 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:03:13.492751 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:13.492760 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:13.492975 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:13.493140 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:13.495331 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:13.497359 (+ 2028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:13.499257 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:13.500116 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:13.500236 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:13.506770 (+ 6534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:13.506785 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:13.506800 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:13.506848 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:13.510503 (+ 3655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:13.511107 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:13.511114 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:13.511116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:13.511244 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:13.511677 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:13.511685 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:13.511689 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:13.512172 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:13.512391 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:13.512885 (+ 494us) server_negotiation.cc:299] Negotiation successful
0812 07:03:13.513151 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13997,"thread_start_us":14808,"threads_started":1}
I20260812 07:03:13.514792 23896 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:13.595701 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 23645
I20260812 07:03:13.618412 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:41935
--local_ip_for_outbound_sockets=127.11.192.195
--tserver_master_addrs=127.11.192.254:45769
--webserver_port=32995
--webserver_interface=127.11.192.195
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.195.keytab
--principal=oryx/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:36469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.192.254:45769,127.11.192.253:41945
--tserver_master_addrs=127.11.192.254:45769,127.11.192.253:41945,127.11.192.252:36667 with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:03:13.923158 24832 heartbeater.cc:503] Master 127.11.192.254:45769 was elected leader, sending a full tablet report...
W20260812 07:03:13.939075 25114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:13.939320 25114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:13.939419 25114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:13.946476 25114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:13.946568 25114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:13.946681 25114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:03:13.955991 25114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:36469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.195.keytab
--principal=oryx/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:41935
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=32995
--tserver_master_addrs=127.11.192.254:45769,127.11.192.253:41945,127.11.192.252:36667
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:13.957283 25114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:13.958500 25114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:13.982142 25131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:13.982259 25132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:13.984879 25134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:13.985183 25114 server_base.cc:1034] running on GCE node
I20260812 07:03:13.986792 25114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:13.988786 25114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:13.990096 25114 hybrid_clock.cc:648] HybridClock initialized: now 1786518193989967 us; error 119 us; skew 500 ppm
I20260812 07:03:13.990684 25114 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:13 dist-test-slave-lgr5 krb5kdc[22890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518193, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:13.996623 25114 init.cc:377] Logged in from keytab as oryx/127.11.192.195@KRBTEST.COM (short username oryx)
I20260812 07:03:13.999636 25114 webserver.cc:460] Webserver started at http://127.11.192.195:32995/ using document root <none> and password file <none>
I20260812 07:03:14.000960 25114 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:14.001291 25114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:14.013257 25114 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.003s sys 0.004s
I20260812 07:03:14.018846 25153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:14.022212 25114 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.006s sys 0.000s
I20260812 07:03:14.022656 25114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "476ffa1a46d444158f741ef6f66efada"
format_stamp: "Formatted at 2026-08-12 07:03:10 on dist-test-slave-lgr5"
server_key: "201d6f6163bf55e312d5080f5b0dc52e"
server_key_iv: "8d2ada8b316ce552d59540d47a7a11d9"
server_key_version: "encryptionkey@0"
I20260812 07:03:14.025511 25114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:14.069837 25114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:14.076908 25114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:14.077325 25114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:14.078936 25114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:14.081543 25114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:14.081691 25114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:14.081878 25114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:14.082026 25114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:14.140914 25114 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:41935
I20260812 07:03:14.140995 25363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:41935 every 8 connection(s)
I20260812 07:03:14.143616 25114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:03:14.153360 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 25114
Aug 12 07:03:14 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518193, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.253@KRBTEST.COM
Aug 12 07:03:14 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518193, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM
I20260812 07:03:14.250885 25370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:14.150130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:55267 (local address 127.11.192.254:45769)
0812 07:03:14.174890 (+ 24760us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:14.174901 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:14.174944 (+ 43us) server_negotiation.cc:407] Connection header received
0812 07:03:14.175086 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:14.175094 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:14.175205 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:14.175412 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:14.177849 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.179981 (+ 2132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.194508 (+ 14527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.195826 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.196058 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:14.227022 (+ 30964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:14.227031 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:14.227035 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:14.227117 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:14.238981 (+ 11864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:14.242473 (+ 3492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:14.242493 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:14.242506 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:14.243600 (+ 1094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:14.245882 (+ 2282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:14.245889 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:14.245892 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:14.246575 (+ 683us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:14.246833 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:14.249699 (+ 2866us) server_negotiation.cc:299] Negotiation successful
0812 07:03:14.250162 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19399,"thread_start_us":19757,"threads_started":1}
I20260812 07:03:14.252328 25371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:14.149937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.11.192.195:55267)
0812 07:03:14.161151 (+ 11214us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:14.161264 (+ 113us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:14.166636 (+ 5372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:14.175821 (+ 9185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:14.175826 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:14.176810 (+ 984us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:14.177410 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:14.177440 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.184253 (+ 6813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.184261 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:14.194143 (+ 9882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:14.194182 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.202509 (+ 8327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.202528 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:14.203220 (+ 692us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:14.214363 (+ 11143us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:14.214422 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:14.226676 (+ 12254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:14.239238 (+ 12562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:14.239253 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:14.239294 (+ 41us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:14.240364 (+ 1070us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:14.243804 (+ 3440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:14.243820 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:14.243824 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:14.244307 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:14.246848 (+ 2541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:14.246874 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:14.247823 (+ 949us) client_negotiation.cc:769] Sending connection context
0812 07:03:14.248459 (+ 636us) client_negotiation.cc:240] Negotiation successful
0812 07:03:14.249521 (+ 1062us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10640,"thread_start_us":10251,"threads_started":1}
I20260812 07:03:14.255379 25365 heartbeater.cc:347] Connected to a master server at 127.11.192.254:45769
I20260812 07:03:14.256042 25365 heartbeater.cc:464] Registering TS with master...
Aug 12 07:03:14 dist-test-slave-lgr5 krb5kdc[22890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518193, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.252@KRBTEST.COM
I20260812 07:03:14.257939 25365 heartbeater.cc:511] Master 127.11.192.254:45769 requested a full tablet report, sending...
I20260812 07:03:14.258463 25372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:14.150059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41945 (local address 127.11.192.195:36747)
0812 07:03:14.168632 (+ 18573us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:14.168668 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:14.168945 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:14.170199 (+ 1254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:14.170226 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:14.172032 (+ 1806us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:14.174197 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:14.174272 (+ 75us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.180393 (+ 6121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.180411 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:14.204831 (+ 24420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:14.204872 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.207248 (+ 2376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.207267 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:14.207618 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:14.211478 (+ 3860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:14.211568 (+ 90us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:14.224651 (+ 13083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:14.240004 (+ 15353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:14.240021 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:14.240025 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:14.242622 (+ 2597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:14.253513 (+ 10891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:14.253529 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:14.253533 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:14.253880 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:14.256390 (+ 2510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:14.256395 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:14.256689 (+ 294us) client_negotiation.cc:769] Sending connection context
0812 07:03:14.257056 (+ 367us) client_negotiation.cc:240] Negotiation successful
0812 07:03:14.257584 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18323,"spinlock_wait_cycles":116480,"thread_start_us":25577,"threads_started":1}
I20260812 07:03:14.259008 25369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:14.148932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:36747 (local address 127.11.192.253:41945)
0812 07:03:14.149785 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:14.149792 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:14.168826 (+ 19034us) server_negotiation.cc:407] Connection header received
0812 07:03:14.169230 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:14.169249 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:14.169452 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:14.169660 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:14.176245 (+ 6585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.178375 (+ 2130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.205115 (+ 26740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.206508 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.206768 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:14.225065 (+ 18297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:14.225083 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:14.225091 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:14.225204 (+ 113us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:14.231899 (+ 6695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:14.242891 (+ 10992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:14.242900 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:14.242903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:14.243237 (+ 334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:14.254126 (+ 10889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:14.254133 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:14.254137 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:14.254932 (+ 795us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:14.255302 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:14.257798 (+ 2496us) server_negotiation.cc:299] Negotiation successful
0812 07:03:14.258249 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":1722,"threads_started":1}
I20260812 07:03:14.260383 25366 heartbeater.cc:347] Connected to a master server at 127.11.192.253:41945
I20260812 07:03:14.260645 25366 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:14.261515 25366 heartbeater.cc:511] Master 127.11.192.253:41945 requested a full tablet report, sending...
I20260812 07:03:14.262812 22968 ts_manager.cc:194] Re-registered known tserver with Master: 476ffa1a46d444158f741ef6f66efada (127.11.192.195:41935)
I20260812 07:03:14.264884 22968 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:55267
I20260812 07:03:14.267606 24170 ts_manager.cc:194] Registered new tserver with Master: 476ffa1a46d444158f741ef6f66efada (127.11.192.195:41935)
I20260812 07:03:14.271071 25392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:14.189597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46706 (local address 127.11.192.253:41945)
0812 07:03:14.217272 (+ 27675us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:14.217289 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:14.217320 (+ 31us) server_negotiation.cc:407] Connection header received
0812 07:03:14.217407 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:14.217423 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:14.217547 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:14.217741 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:14.223965 (+ 6224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.229138 (+ 5173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.233417 (+ 4279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.234650 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.234864 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:14.248741 (+ 13877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:14.248751 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:14.248756 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:14.248833 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:14.258422 (+ 9589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:14.266556 (+ 8134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:14.266574 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:14.266577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:14.266937 (+ 360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:14.267823 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:14.267840 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:14.267844 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:14.268561 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:14.269022 (+ 461us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:14.269857 (+ 835us) server_negotiation.cc:299] Negotiation successful
0812 07:03:14.270235 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27449,"thread_start_us":27189,"threads_started":1}
I20260812 07:03:14.278700 25383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:14.176736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:36667 (local address 127.11.192.195:44099)
0812 07:03:14.206664 (+ 29928us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:14.206706 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:14.206903 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:14.215751 (+ 8848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:14.215757 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:14.218253 (+ 2496us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:14.218948 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:14.218976 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.231824 (+ 12848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.231841 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:14.234577 (+ 2736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:14.234611 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.242967 (+ 8356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.243011 (+ 44us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:14.243361 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:14.245106 (+ 1745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:14.245146 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:14.259938 (+ 14792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:14.273884 (+ 13946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:14.273892 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:14.273907 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:14.274909 (+ 1002us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:14.275762 (+ 853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:14.275765 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:14.275768 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:14.275983 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:14.276838 (+ 855us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:14.276845 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:14.277222 (+ 377us) client_negotiation.cc:769] Sending connection context
0812 07:03:14.277570 (+ 348us) client_negotiation.cc:240] Negotiation successful
0812 07:03:14.278130 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29665,"thread_start_us":29425,"threads_started":1}
I20260812 07:03:14.280236 25388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:14.186114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44224 (local address 127.11.192.254:45769)
0812 07:03:14.220940 (+ 34826us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:14.220948 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:14.221009 (+ 61us) server_negotiation.cc:407] Connection header received
0812 07:03:14.221104 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:14.221120 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:14.221250 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:14.221439 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:14.228412 (+ 6973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.230314 (+ 1902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.245744 (+ 15430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.246902 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.247186 (+ 284us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:14.270402 (+ 23216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:14.270421 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:14.270427 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:14.270504 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:14.274858 (+ 4354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:14.276017 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:14.276035 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:14.276039 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:14.276345 (+ 306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:14.277178 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:14.277185 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:14.277188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:14.277857 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:14.278256 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:14.278967 (+ 711us) server_negotiation.cc:299] Negotiation successful
0812 07:03:14.279402 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34594,"thread_start_us":34376,"threads_started":1}
I20260812 07:03:14.280524 25395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:14.176980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:44099 (local address 127.11.192.252:36667)
0812 07:03:14.209738 (+ 32758us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:14.209749 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:14.209806 (+ 57us) server_negotiation.cc:407] Connection header received
0812 07:03:14.213603 (+ 3797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:14.213611 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:14.213765 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:14.213980 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:14.220221 (+ 6241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.222091 (+ 1870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.236681 (+ 14590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.240615 (+ 3934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.240840 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:14.260945 (+ 20105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:14.260967 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:14.260972 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:14.261068 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:14.272502 (+ 11434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:14.275212 (+ 2710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:14.275219 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:14.275222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:14.275528 (+ 306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:14.276183 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:14.276189 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:14.276204 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:14.276643 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:14.276848 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:14.279309 (+ 2461us) server_negotiation.cc:299] Negotiation successful
0812 07:03:14.279671 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32515,"thread_start_us":49928,"threads_started":1}
I20260812 07:03:14.282199 25364 heartbeater.cc:347] Connected to a master server at 127.11.192.252:36667
I20260812 07:03:14.282260 25384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:14.194071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38988 (local address 127.11.192.252:36667)
0812 07:03:14.219736 (+ 25665us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:14.219754 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:14.219787 (+ 33us) server_negotiation.cc:407] Connection header received
0812 07:03:14.219890 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:14.219896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:14.227120 (+ 7224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:14.227321 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:14.235014 (+ 7693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.236944 (+ 1930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.241796 (+ 4852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.243197 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.243414 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:14.264718 (+ 21304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:14.264739 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:14.264743 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:14.264828 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:14.275544 (+ 10716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:14.276892 (+ 1348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:14.276911 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:14.276914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:14.277249 (+ 335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:14.277878 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:14.277891 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:14.277894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:14.278551 (+ 657us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:14.281171 (+ 2620us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:14.281432 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 07:03:14.281793 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25469,"thread_start_us":19011,"threads_started":1}
I20260812 07:03:14.282433 25364 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:14.283181 25364 heartbeater.cc:511] Master 127.11.192.252:36667 requested a full tablet report, sending...
I20260812 07:03:14.285094 24501 ts_manager.cc:194] Registered new tserver with Master: 476ffa1a46d444158f741ef6f66efada (127.11.192.195:41935)
I20260812 07:03:14.288175 22968 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:03:14.300356 22968 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:03:14.351140 25425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:14.333556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58248 (local address 127.11.192.193:42031)
0812 07:03:14.334547 (+ 991us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:14.334562 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:14.334625 (+ 63us) server_negotiation.cc:407] Connection header received
0812 07:03:14.334829 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:14.334840 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:14.335455 (+ 615us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:14.335835 (+ 380us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:14.337617 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.340633 (+ 3016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.345498 (+ 4865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.349007 (+ 3509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.349373 (+ 366us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:14.349789 (+ 416us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:14.350181 (+ 392us) server_negotiation.cc:299] Negotiation successful
0812 07:03:14.350485 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":625,"threads_started":1}
I20260812 07:03:14.351469 25424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:14.332727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:42031 (local address 127.0.0.1:58248)
0812 07:03:14.333700 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:14.333714 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:14.333832 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:14.336367 (+ 2535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:14.336376 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:14.336441 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:14.337348 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:14.337372 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.340946 (+ 3574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.340954 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:14.345086 (+ 4132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:14.345115 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.349408 (+ 4293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.349417 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:14.349808 (+ 391us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:14.349877 (+ 69us) client_negotiation.cc:769] Sending connection context
0812 07:03:14.350074 (+ 197us) client_negotiation.cc:240] Negotiation successful
0812 07:03:14.350404 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":62,"thread_start_us":481,"threads_started":1}
I20260812 07:03:14.352845 25418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:14.331962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:41935 (local address 127.0.0.1:39902)
0812 07:03:14.333053 (+ 1091us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:14.333103 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:14.333328 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:14.339510 (+ 6182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:14.339518 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:14.339584 (+ 66us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:14.340279 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:14.340304 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.344520 (+ 4216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.344537 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:14.347835 (+ 3298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:14.347859 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.351432 (+ 3573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.351441 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:14.351784 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:14.351859 (+ 75us) client_negotiation.cc:769] Sending connection context
0812 07:03:14.351971 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 07:03:14.352237 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":866,"thread_start_us":568,"threads_started":1}
I20260812 07:03:14.356685 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:14.332491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39902 (local address 127.11.192.195:41935)
0812 07:03:14.338404 (+ 5913us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:14.338417 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:14.338475 (+ 58us) server_negotiation.cc:407] Connection header received
0812 07:03:14.338660 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:14.338669 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:14.339142 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:14.340662 (+ 1520us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:14.341234 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.344240 (+ 3006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.348326 (+ 4086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.351022 (+ 2696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.353410 (+ 2388us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:14.355888 (+ 2478us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:14.355993 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 07:03:14.356259 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5616,"thread_start_us":5322,"threads_started":1}
I20260812 07:03:14.359571 24714 tablet_service.cc:1467] Processing CreateTablet for tablet 8a79fba1173b4bdb9894d9fc13313ebe (DEFAULT_TABLE table=test-table [id=9a5ea3396b2d4079a8cad0ab9ea16d9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:03:14.361943 25427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:14.335222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56204 (local address 127.11.192.194:34145)
0812 07:03:14.343429 (+ 8207us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:14.343438 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:14.343481 (+ 43us) server_negotiation.cc:407] Connection header received
0812 07:03:14.343615 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:14.343623 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:14.343868 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:14.344217 (+ 349us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:14.345663 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.348336 (+ 2673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.352709 (+ 4373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.358273 (+ 5564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.358588 (+ 315us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:14.358931 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:14.361136 (+ 2205us) server_negotiation.cc:299] Negotiation successful
0812 07:03:14.361441 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7967,"thread_start_us":7783,"threads_started":1}
I20260812 07:03:14.362362 25429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:14.335890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:34145 (local address 127.0.0.1:56204)
0812 07:03:14.336990 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:14.337028 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:14.337305 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:14.344194 (+ 6889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:14.344211 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:14.344252 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:14.345278 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:14.345304 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.348629 (+ 3325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.348637 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:14.352289 (+ 3652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:14.352330 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.359299 (+ 6969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.359318 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:14.359646 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:14.359720 (+ 74us) client_negotiation.cc:769] Sending connection context
0812 07:03:14.359873 (+ 153us) client_negotiation.cc:240] Negotiation successful
0812 07:03:14.360159 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"mutex_wait_us":4,"thread_start_us":751,"threads_started":1}
I20260812 07:03:14.362591 24714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a79fba1173b4bdb9894d9fc13313ebe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:14.362206 25249 tablet_service.cc:1467] Processing CreateTablet for tablet 8a79fba1173b4bdb9894d9fc13313ebe (DEFAULT_TABLE table=test-table [id=9a5ea3396b2d4079a8cad0ab9ea16d9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:03:14.364362 25249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a79fba1173b4bdb9894d9fc13313ebe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:14.369572 24963 tablet_service.cc:1467] Processing CreateTablet for tablet 8a79fba1173b4bdb9894d9fc13313ebe (DEFAULT_TABLE table=test-table [id=9a5ea3396b2d4079a8cad0ab9ea16d9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:03:14.372448 24963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a79fba1173b4bdb9894d9fc13313ebe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:14.387727 25430 tablet_bootstrap.cc:492] T 8a79fba1173b4bdb9894d9fc13313ebe P c58ec4c6df7d4bed947f921c427777c7: Bootstrap starting.
I20260812 07:03:14.394006 25430 tablet_bootstrap.cc:654] T 8a79fba1173b4bdb9894d9fc13313ebe P c58ec4c6df7d4bed947f921c427777c7: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:14.398458 25430 log.cc:824] T 8a79fba1173b4bdb9894d9fc13313ebe P c58ec4c6df7d4bed947f921c427777c7: Log is configured to *not* fsync() on all Append() calls
I20260812 07:03:14.402513 25431 tablet_bootstrap.cc:492] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada: Bootstrap starting.
I20260812 07:03:14.409934 25434 tablet_bootstrap.cc:492] T 8a79fba1173b4bdb9894d9fc13313ebe P fcfbae397ccb4992985a1147955cba30: Bootstrap starting.
I20260812 07:03:14.410986 25430 tablet_bootstrap.cc:492] T 8a79fba1173b4bdb9894d9fc13313ebe P c58ec4c6df7d4bed947f921c427777c7: No bootstrap required, opened a new log
I20260812 07:03:14.411458 25430 ts_tablet_manager.cc:1397] T 8a79fba1173b4bdb9894d9fc13313ebe P c58ec4c6df7d4bed947f921c427777c7: Time spent bootstrapping tablet: real 0.024s user 0.014s sys 0.004s
I20260812 07:03:14.413353 25431 tablet_bootstrap.cc:654] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:14.415964 25431 log.cc:824] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada: Log is configured to *not* fsync() on all Append() calls
I20260812 07:03:14.420181 25434 tablet_bootstrap.cc:654] T 8a79fba1173b4bdb9894d9fc13313ebe P fcfbae397ccb4992985a1147955cba30: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:14.422679 25434 log.cc:824] T 8a79fba1173b4bdb9894d9fc13313ebe P fcfbae397ccb4992985a1147955cba30: Log is configured to *not* fsync() on all Append() calls
I20260812 07:03:14.428791 25431 tablet_bootstrap.cc:492] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada: No bootstrap required, opened a new log
I20260812 07:03:14.428779 25434 tablet_bootstrap.cc:492] T 8a79fba1173b4bdb9894d9fc13313ebe P fcfbae397ccb4992985a1147955cba30: No bootstrap required, opened a new log
I20260812 07:03:14.429450 25434 ts_tablet_manager.cc:1397] T 8a79fba1173b4bdb9894d9fc13313ebe P fcfbae397ccb4992985a1147955cba30: Time spent bootstrapping tablet: real 0.020s user 0.014s sys 0.000s
I20260812 07:03:14.429514 25431 ts_tablet_manager.cc:1397] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada: Time spent bootstrapping tablet: real 0.027s user 0.022s sys 0.000s
I20260812 07:03:14.431205 25430 raft_consensus.cc:348] T 8a79fba1173b4bdb9894d9fc13313ebe P c58ec4c6df7d4bed947f921c427777c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "476ffa1a46d444158f741ef6f66efada" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41935 } } peers { permanent_uuid: "c58ec4c6df7d4bed947f921c427777c7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 42031 } } peers { permanent_uuid: "fcfbae397ccb4992985a1147955cba30" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34145 } }
I20260812 07:03:14.431838 25430 raft_consensus.cc:374] T 8a79fba1173b4bdb9894d9fc13313ebe P c58ec4c6df7d4bed947f921c427777c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:14.432017 25430 raft_consensus.cc:729] T 8a79fba1173b4bdb9894d9fc13313ebe P c58ec4c6df7d4bed947f921c427777c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c58ec4c6df7d4bed947f921c427777c7, State: Initialized, Role: FOLLOWER
I20260812 07:03:14.434366 25430 consensus_queue.cc:260] T 8a79fba1173b4bdb9894d9fc13313ebe P c58ec4c6df7d4bed947f921c427777c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "476ffa1a46d444158f741ef6f66efada" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41935 } } peers { permanent_uuid: "c58ec4c6df7d4bed947f921c427777c7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 42031 } } peers { permanent_uuid: "fcfbae397ccb4992985a1147955cba30" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34145 } }
I20260812 07:03:14.437566 25430 ts_tablet_manager.cc:1428] T 8a79fba1173b4bdb9894d9fc13313ebe P c58ec4c6df7d4bed947f921c427777c7: Time spent starting tablet: real 0.026s user 0.023s sys 0.000s
I20260812 07:03:14.451128 25431 raft_consensus.cc:348] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "476ffa1a46d444158f741ef6f66efada" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41935 } } peers { permanent_uuid: "c58ec4c6df7d4bed947f921c427777c7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 42031 } } peers { permanent_uuid: "fcfbae397ccb4992985a1147955cba30" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34145 } }
I20260812 07:03:14.452204 25431 raft_consensus.cc:374] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:14.451674 25434 raft_consensus.cc:348] T 8a79fba1173b4bdb9894d9fc13313ebe P fcfbae397ccb4992985a1147955cba30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "476ffa1a46d444158f741ef6f66efada" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41935 } } peers { permanent_uuid: "c58ec4c6df7d4bed947f921c427777c7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 42031 } } peers { permanent_uuid: "fcfbae397ccb4992985a1147955cba30" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34145 } }
I20260812 07:03:14.452447 25431 raft_consensus.cc:729] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 476ffa1a46d444158f741ef6f66efada, State: Initialized, Role: FOLLOWER
I20260812 07:03:14.452658 25434 raft_consensus.cc:374] T 8a79fba1173b4bdb9894d9fc13313ebe P fcfbae397ccb4992985a1147955cba30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:14.452853 25434 raft_consensus.cc:729] T 8a79fba1173b4bdb9894d9fc13313ebe P fcfbae397ccb4992985a1147955cba30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcfbae397ccb4992985a1147955cba30, State: Initialized, Role: FOLLOWER
I20260812 07:03:14.453775 25431 consensus_queue.cc:260] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "476ffa1a46d444158f741ef6f66efada" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41935 } } peers { permanent_uuid: "c58ec4c6df7d4bed947f921c427777c7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 42031 } } peers { permanent_uuid: "fcfbae397ccb4992985a1147955cba30" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34145 } }
I20260812 07:03:14.454125 25434 consensus_queue.cc:260] T 8a79fba1173b4bdb9894d9fc13313ebe P fcfbae397ccb4992985a1147955cba30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "476ffa1a46d444158f741ef6f66efada" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41935 } } peers { permanent_uuid: "c58ec4c6df7d4bed947f921c427777c7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 42031 } } peers { permanent_uuid: "fcfbae397ccb4992985a1147955cba30" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34145 } }
I20260812 07:03:14.455948 25365 heartbeater.cc:503] Master 127.11.192.254:45769 was elected leader, sending a full tablet report...
I20260812 07:03:14.457692 25431 ts_tablet_manager.cc:1428] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada: Time spent starting tablet: real 0.028s user 0.022s sys 0.005s
I20260812 07:03:14.463837 25434 ts_tablet_manager.cc:1428] T 8a79fba1173b4bdb9894d9fc13313ebe P fcfbae397ccb4992985a1147955cba30: Time spent starting tablet: real 0.034s user 0.030s sys 0.000s
I20260812 07:03:14.464330 25091 heartbeater.cc:503] Master 127.11.192.254:45769 was elected leader, sending a full tablet report...
W20260812 07:03:14.470836 25095 tablet.cc:2369] T 8a79fba1173b4bdb9894d9fc13313ebe P fcfbae397ccb4992985a1147955cba30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:03:14.541689 25441 raft_consensus.cc:482] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:03:14.542162 25441 raft_consensus.cc:504] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "476ffa1a46d444158f741ef6f66efada" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41935 } } peers { permanent_uuid: "c58ec4c6df7d4bed947f921c427777c7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 42031 } } peers { permanent_uuid: "fcfbae397ccb4992985a1147955cba30" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34145 } }
I20260812 07:03:14.545151 25441 leader_election.cc:288] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c58ec4c6df7d4bed947f921c427777c7 (127.11.192.193:42031), fcfbae397ccb4992985a1147955cba30 (127.11.192.194:34145)
I20260812 07:03:14.556226 25383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:14.545461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:42031 (local address 127.11.192.195:40535)
0812 07:03:14.545773 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:14.545808 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:14.546057 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:14.546577 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:14.546587 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:14.546644 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:14.547004 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:14.547025 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.549348 (+ 2323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.549353 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:14.552557 (+ 3204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:14.552602 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.555206 (+ 2604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.555215 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:14.555478 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:14.555525 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 07:03:14.555666 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 07:03:14.555832 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 07:03:14.556473 25425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:14.545618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:40535 (local address 127.11.192.193:42031)
0812 07:03:14.545965 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:14.545984 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:14.546031 (+ 47us) server_negotiation.cc:407] Connection header received
0812 07:03:14.546269 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:14.546273 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:14.546364 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:14.546523 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:14.547372 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.549154 (+ 1782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.552809 (+ 3655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.554898 (+ 2089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.555240 (+ 342us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:14.555399 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:14.555752 (+ 353us) server_negotiation.cc:299] Negotiation successful
0812 07:03:14.555998 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 07:03:14.557385 25372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:14.546526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:34145 (local address 127.11.192.195:42997)
0812 07:03:14.546855 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:14.546895 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:14.547087 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:14.547921 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:14.547925 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:14.547958 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:14.548412 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:14.548441 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.550355 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.550363 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:14.553446 (+ 3083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:14.553481 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.556321 (+ 2840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.556339 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:14.556714 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:14.556759 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 07:03:14.556852 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 07:03:14.557008 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 07:03:14.557721 25427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:14.547012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:42997 (local address 127.11.192.194:34145)
0812 07:03:14.547383 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:14.547389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:14.547430 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:03:14.547577 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:14.547593 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:14.547735 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:14.547901 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:14.548685 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.550130 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.553791 (+ 3661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.556049 (+ 2258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.556821 (+ 772us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:14.556982 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:14.557070 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 07:03:14.557321 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260812 07:03:14.558079 24747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a79fba1173b4bdb9894d9fc13313ebe" candidate_uuid: "476ffa1a46d444158f741ef6f66efada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c58ec4c6df7d4bed947f921c427777c7" is_pre_election: true
I20260812 07:03:14.559036 24747 raft_consensus.cc:2454] T 8a79fba1173b4bdb9894d9fc13313ebe P c58ec4c6df7d4bed947f921c427777c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 476ffa1a46d444158f741ef6f66efada in term 0.
I20260812 07:03:14.559212 25001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a79fba1173b4bdb9894d9fc13313ebe" candidate_uuid: "476ffa1a46d444158f741ef6f66efada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcfbae397ccb4992985a1147955cba30" is_pre_election: true
I20260812 07:03:14.560206 25001 raft_consensus.cc:2454] T 8a79fba1173b4bdb9894d9fc13313ebe P fcfbae397ccb4992985a1147955cba30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 476ffa1a46d444158f741ef6f66efada in term 0.
I20260812 07:03:14.560346 25166 leader_election.cc:302] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 476ffa1a46d444158f741ef6f66efada, c58ec4c6df7d4bed947f921c427777c7; no voters:
I20260812 07:03:14.561290 25441 raft_consensus.cc:2781] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:03:14.561569 25441 raft_consensus.cc:482] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:03:14.561802 25441 raft_consensus.cc:3037] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:14.565680 25441 raft_consensus.cc:504] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "476ffa1a46d444158f741ef6f66efada" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41935 } } peers { permanent_uuid: "c58ec4c6df7d4bed947f921c427777c7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 42031 } } peers { permanent_uuid: "fcfbae397ccb4992985a1147955cba30" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34145 } }
I20260812 07:03:14.566596 25441 leader_election.cc:288] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada [CANDIDATE]: Term 1 election: Requested vote from peers c58ec4c6df7d4bed947f921c427777c7 (127.11.192.193:42031), fcfbae397ccb4992985a1147955cba30 (127.11.192.194:34145)
I20260812 07:03:14.567078 24747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a79fba1173b4bdb9894d9fc13313ebe" candidate_uuid: "476ffa1a46d444158f741ef6f66efada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c58ec4c6df7d4bed947f921c427777c7"
I20260812 07:03:14.567267 25001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a79fba1173b4bdb9894d9fc13313ebe" candidate_uuid: "476ffa1a46d444158f741ef6f66efada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcfbae397ccb4992985a1147955cba30"
I20260812 07:03:14.567349 24747 raft_consensus.cc:3037] T 8a79fba1173b4bdb9894d9fc13313ebe P c58ec4c6df7d4bed947f921c427777c7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:14.567576 25001 raft_consensus.cc:3037] T 8a79fba1173b4bdb9894d9fc13313ebe P fcfbae397ccb4992985a1147955cba30 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:14.570951 24747 raft_consensus.cc:2454] T 8a79fba1173b4bdb9894d9fc13313ebe P c58ec4c6df7d4bed947f921c427777c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 476ffa1a46d444158f741ef6f66efada in term 1.
I20260812 07:03:14.571169 25001 raft_consensus.cc:2454] T 8a79fba1173b4bdb9894d9fc13313ebe P fcfbae397ccb4992985a1147955cba30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 476ffa1a46d444158f741ef6f66efada in term 1.
I20260812 07:03:14.571625 25166 leader_election.cc:302] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 476ffa1a46d444158f741ef6f66efada, c58ec4c6df7d4bed947f921c427777c7; no voters:
I20260812 07:03:14.572252 25441 raft_consensus.cc:2781] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:03:14.572960 25441 raft_consensus.cc:686] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada [term 1 LEADER]: Becoming Leader. State: Replica: 476ffa1a46d444158f741ef6f66efada, State: Running, Role: LEADER
I20260812 07:03:14.573899 25441 consensus_queue.cc:237] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "476ffa1a46d444158f741ef6f66efada" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41935 } } peers { permanent_uuid: "c58ec4c6df7d4bed947f921c427777c7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 42031 } } peers { permanent_uuid: "fcfbae397ccb4992985a1147955cba30" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34145 } }
I20260812 07:03:14.582871 22967 catalog_manager.cc:5495] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada reported cstate change: term changed from 0 to 1, leader changed from <none> to 476ffa1a46d444158f741ef6f66efada (127.11.192.195). New cstate: current_term: 1 leader_uuid: "476ffa1a46d444158f741ef6f66efada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "476ffa1a46d444158f741ef6f66efada" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41935 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c58ec4c6df7d4bed947f921c427777c7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 42031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcfbae397ccb4992985a1147955cba30" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34145 } health_report { overall_health: UNKNOWN } } }
W20260812 07:03:14.632637 24835 tablet.cc:2369] T 8a79fba1173b4bdb9894d9fc13313ebe P c58ec4c6df7d4bed947f921c427777c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:03:14.651809 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:14.642668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39910 (local address 127.11.192.195:41935)
0812 07:03:14.642978 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:14.642985 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:14.643013 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:03:14.643322 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:14.643339 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:14.643501 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:14.643716 (+ 215us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:03:14.644355 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.645479 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.647763 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.648600 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.648814 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:14.649693 (+ 879us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:03:14.650817 (+ 1124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:03:14.651100 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:14.651191 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 07:03:14.651358 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
W20260812 07:03:14.653674 25368 tablet.cc:2369] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:03:14.665110 24747 raft_consensus.cc:1261] T 8a79fba1173b4bdb9894d9fc13313ebe P c58ec4c6df7d4bed947f921c427777c7 [term 1 FOLLOWER]: Refusing update from remote peer 476ffa1a46d444158f741ef6f66efada: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:03:14.665391 25001 raft_consensus.cc:1261] T 8a79fba1173b4bdb9894d9fc13313ebe P fcfbae397ccb4992985a1147955cba30 [term 1 FOLLOWER]: Refusing update from remote peer 476ffa1a46d444158f741ef6f66efada: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:03:14.666600 25444 consensus_queue.cc:1035] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada [LEADER]: Connected to new peer: Peer: permanent_uuid: "c58ec4c6df7d4bed947f921c427777c7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 42031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:03:14.667151 25441 consensus_queue.cc:1035] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcfbae397ccb4992985a1147955cba30" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:03:14.719822 22968 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:03:14.720268 22968 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:03:14.731240 22968 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f729c2efda9346f2a36c4ba13c5c56f9: Sending DeleteTablet for 3 replicas of tablet 8a79fba1173b4bdb9894d9fc13313ebe
I20260812 07:03:14.733109 25249 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a79fba1173b4bdb9894d9fc13313ebe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:03:14 UTC) from {username='oryx', principal='oryx/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:39902
I20260812 07:03:14.733714 24714 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a79fba1173b4bdb9894d9fc13313ebe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:03:14 UTC) from {username='oryx', principal='oryx/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:58248
I20260812 07:03:14.737582 24963 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a79fba1173b4bdb9894d9fc13313ebe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:03:14 UTC) from {username='oryx', principal='oryx/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:56204
I20260812 07:03:14.737819 25471 tablet_replica.cc:330] T 8a79fba1173b4bdb9894d9fc13313ebe P c58ec4c6df7d4bed947f921c427777c7: stopping tablet replica
I20260812 07:03:14.738718 25471 raft_consensus.cc:2229] T 8a79fba1173b4bdb9894d9fc13313ebe P c58ec4c6df7d4bed947f921c427777c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:03:14.739774 25471 raft_consensus.cc:2258] T 8a79fba1173b4bdb9894d9fc13313ebe P c58ec4c6df7d4bed947f921c427777c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:03:14.744134 25470 tablet_replica.cc:330] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada: stopping tablet replica
I20260812 07:03:14.744253 25471 ts_tablet_manager.cc:1905] T 8a79fba1173b4bdb9894d9fc13313ebe P c58ec4c6df7d4bed947f921c427777c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:03:14.745002 25470 raft_consensus.cc:2229] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:03:14.746220 25470 raft_consensus.cc:2258] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:03:14.755491 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 22909
I20260812 07:03:14.757398 23042 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:14.759414 25471 ts_tablet_manager.cc:1918] T 8a79fba1173b4bdb9894d9fc13313ebe P c58ec4c6df7d4bed947f921c427777c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:03:14.759950 25471 log.cc:1196] T 8a79fba1173b4bdb9894d9fc13313ebe P c58ec4c6df7d4bed947f921c427777c7: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-0/wal/wals/8a79fba1173b4bdb9894d9fc13313ebe
I20260812 07:03:14.761790 25471 ts_tablet_manager.cc:1939] T 8a79fba1173b4bdb9894d9fc13313ebe P c58ec4c6df7d4bed947f921c427777c7: Deleting consensus metadata
I20260812 07:03:14.762125 25470 ts_tablet_manager.cc:1905] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:03:14.769099 25472 tablet_replica.cc:330] T 8a79fba1173b4bdb9894d9fc13313ebe P fcfbae397ccb4992985a1147955cba30: stopping tablet replica
I20260812 07:03:14.773555 25472 raft_consensus.cc:2229] T 8a79fba1173b4bdb9894d9fc13313ebe P fcfbae397ccb4992985a1147955cba30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:03:14.774626 25472 raft_consensus.cc:2258] T 8a79fba1173b4bdb9894d9fc13313ebe P fcfbae397ccb4992985a1147955cba30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:03:14.778656 25472 ts_tablet_manager.cc:1905] T 8a79fba1173b4bdb9894d9fc13313ebe P fcfbae397ccb4992985a1147955cba30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:03:14.784780 25470 ts_tablet_manager.cc:1918] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:03:14.785050 25470 log.cc:1196] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/8a79fba1173b4bdb9894d9fc13313ebe
I20260812 07:03:14.786142 25470 ts_tablet_manager.cc:1939] T 8a79fba1173b4bdb9894d9fc13313ebe P 476ffa1a46d444158f741ef6f66efada: Deleting consensus metadata
I20260812 07:03:14.794065 25472 ts_tablet_manager.cc:1918] T 8a79fba1173b4bdb9894d9fc13313ebe P fcfbae397ccb4992985a1147955cba30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:03:14.794353 25472 log.cc:1196] T 8a79fba1173b4bdb9894d9fc13313ebe P fcfbae397ccb4992985a1147955cba30: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/8a79fba1173b4bdb9894d9fc13313ebe
I20260812 07:03:14.795401 25472 ts_tablet_manager.cc:1939] T 8a79fba1173b4bdb9894d9fc13313ebe P fcfbae397ccb4992985a1147955cba30: Deleting consensus metadata
I20260812 07:03:14.802553 25392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:14.742832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46710 (local address 127.11.192.253:41945)
0812 07:03:14.743091 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:14.743101 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:14.754924 (+ 11823us) server_negotiation.cc:407] Connection header received
0812 07:03:14.755894 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:14.755901 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:14.756056 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:14.756256 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:14.773153 (+ 16897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.775165 (+ 2012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.778738 (+ 3573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.779883 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.780144 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:14.787752 (+ 7608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:14.787774 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:14.787780 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:14.788000 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:14.795908 (+ 7908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:14.797537 (+ 1629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:14.797554 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:14.797558 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:14.797873 (+ 315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:14.799077 (+ 1204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:14.799084 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:14.799087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:14.799915 (+ 828us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:14.800439 (+ 524us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:14.801322 (+ 883us) server_negotiation.cc:299] Negotiation successful
0812 07:03:14.801892 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:03:14.833755 25384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:14.753644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38996 (local address 127.11.192.252:36667)
0812 07:03:14.754072 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:14.754080 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:14.802666 (+ 48586us) server_negotiation.cc:407] Connection header received
0812 07:03:14.803083 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:14.803099 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:14.803239 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:14.803424 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:14.807213 (+ 3789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.809425 (+ 2212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.813452 (+ 4027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:14.814797 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:14.814968 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:14.822016 (+ 7048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:14.822034 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:14.822051 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:14.822127 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:14.828323 (+ 6196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:14.829830 (+ 1507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:14.829847 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:14.829851 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:14.830090 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:14.830865 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:14.830873 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:14.830876 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:14.831460 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:14.831849 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:14.832779 (+ 930us) server_negotiation.cc:299] Negotiation successful
0812 07:03:14.833166 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172}
I20260812 07:03:14.971652 22938 catalog_manager.cc:4841] TS fcfbae397ccb4992985a1147955cba30 (127.11.192.194:34145): tablet 8a79fba1173b4bdb9894d9fc13313ebe (table test-table [id=9a5ea3396b2d4079a8cad0ab9ea16d9e]) successfully deleted
I20260812 07:03:14.972795 22938 catalog_manager.cc:4841] TS c58ec4c6df7d4bed947f921c427777c7 (127.11.192.193:42031): tablet 8a79fba1173b4bdb9894d9fc13313ebe (table test-table [id=9a5ea3396b2d4079a8cad0ab9ea16d9e]) successfully deleted
I20260812 07:03:14.972995 22933 catalog_manager.cc:4841] TS 476ffa1a46d444158f741ef6f66efada (127.11.192.195:41935): tablet 8a79fba1173b4bdb9894d9fc13313ebe (table test-table [id=9a5ea3396b2d4079a8cad0ab9ea16d9e]) successfully deleted
I20260812 07:03:14.973650 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 22909
I20260812 07:03:15.081562 25384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:15.052436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39008 (local address 127.11.192.252:36667)
0812 07:03:15.052875 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:15.052886 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:15.052930 (+ 44us) server_negotiation.cc:407] Connection header received
0812 07:03:15.053070 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:15.053088 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:15.053305 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:15.053516 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:15.058771 (+ 5255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:15.060967 (+ 2196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:15.064665 (+ 3698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:15.066091 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:15.066345 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:15.071195 (+ 4850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:15.071201 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:15.071205 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:15.071284 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:15.076378 (+ 5094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:15.077405 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:15.077421 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:15.077424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:15.077727 (+ 303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:15.078517 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:15.078534 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:15.078540 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:15.079183 (+ 643us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:15.079600 (+ 417us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:15.080460 (+ 860us) server_negotiation.cc:299] Negotiation successful
0812 07:03:15.081000 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192}
I20260812 07:03:15.082095 25392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:15.048579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46718 (local address 127.11.192.253:41945)
0812 07:03:15.049078 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:15.049089 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:15.051078 (+ 1989us) server_negotiation.cc:407] Connection header received
0812 07:03:15.051582 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:15.051600 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:15.051775 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:15.051878 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:15.056383 (+ 4505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:15.058483 (+ 2100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:15.060932 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:15.061724 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:15.061962 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:15.067591 (+ 5629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:15.067600 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:15.067617 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:15.067699 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:15.074812 (+ 7113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:15.076175 (+ 1363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:15.076182 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:15.076186 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:15.076436 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:15.078058 (+ 1622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:15.078063 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:15.078074 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:15.078629 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:15.079059 (+ 430us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:15.080963 (+ 1904us) server_negotiation.cc:299] Negotiation successful
0812 07:03:15.081412 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260812 07:03:15.699748 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:15.698541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.11.192.193:48131)
0812 07:03:15.699371 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:15.699554 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":498,"threads_started":1}
W20260812 07:03:15.700660 24832 heartbeater.cc:644] Failed to heartbeat to 127.11.192.254:45769 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
I20260812 07:03:15.712055 25496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:15.710226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.11.192.194:51715)
0812 07:03:15.711379 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:15.711616 (+ 237us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":973,"thread_start_us":390,"threads_started":1}
W20260812 07:03:15.714093 25091 heartbeater.cc:644] Failed to heartbeat to 127.11.192.254:45769 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
I20260812 07:03:15.714673 25497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:15.712828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.11.192.195:38471)
0812 07:03:15.714004 (+ 1176us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:15.714326 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":806,"thread_start_us":689,"threads_started":1}
W20260812 07:03:15.715195 25365 heartbeater.cc:644] Failed to heartbeat to 127.11.192.254:45769 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
I20260812 07:03:16.231535 25498 raft_consensus.cc:482] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f729c2efda9346f2a36c4ba13c5c56f9)
I20260812 07:03:16.231822 25498 raft_consensus.cc:504] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: false } }
I20260812 07:03:16.233182 25498 leader_election.cc:288] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f729c2efda9346f2a36c4ba13c5c56f9 (127.11.192.254:45769), 8d82aaf8d7ef451dabbed9397999081c (127.11.192.253:41945)
I20260812 07:03:16.234478 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.233346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.0.0.1:44622)
0812 07:03:16.233925 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.234161 (+ 236us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":264,"threads_started":1}
I20260812 07:03:16.234884 25502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f729c2efda9346f2a36c4ba13c5c56f9)
W20260812 07:03:16.234864 24466 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
I20260812 07:03:16.235146 25502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: false } }
I20260812 07:03:16.236557 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.236119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.0.0.1:44632)
0812 07:03:16.236264 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.236364 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 07:03:16.236912 24466 leader_election.cc:334] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f729c2efda9346f2a36c4ba13c5c56f9 (127.11.192.254:45769): Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
I20260812 07:03:16.237563 25502 leader_election.cc:288] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f729c2efda9346f2a36c4ba13c5c56f9 (127.11.192.254:45769), 998954c32f6b45cba1277aacf2dc5196 (127.11.192.252:36667)
I20260812 07:03:16.237988 25505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.236847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.0.0.1:44644)
0812 07:03:16.237437 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.237684 (+ 247us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":269,"threads_started":1}
W20260812 07:03:16.238394 24149 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
I20260812 07:03:16.243506 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.233361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41945 (local address 127.0.0.1:46726)
0812 07:03:16.234217 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.234241 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:16.234401 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:16.236343 (+ 1942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:16.236348 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:16.236374 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:16.236703 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:16.236718 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.238369 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.238375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:16.240721 (+ 2346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:16.240740 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.242726 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.242733 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:16.242928 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:16.242969 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 07:03:16.243034 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 07:03:16.243171 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":570,"threads_started":1}
I20260812 07:03:16.244789 25501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.233506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46726 (local address 127.11.192.253:41945)
0812 07:03:16.236011 (+ 2505us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:16.236016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:16.236036 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:03:16.236136 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:16.236141 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:16.236216 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:16.236326 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:16.236880 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.238189 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.240964 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.242513 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.244119 (+ 1606us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:16.244202 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:16.244251 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 07:03:16.244383 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":433,"server-negotiator.queue_time_us":2392,"thread_start_us":2202,"threads_started":1}
I20260812 07:03:16.245476 24188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "998954c32f6b45cba1277aacf2dc5196" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8d82aaf8d7ef451dabbed9397999081c" is_pre_election: true
I20260812 07:03:16.245709 25508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.244987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.0.0.1:44650)
0812 07:03:16.245230 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.245426 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260812 07:03:16.245860 25509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.238055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39018 (local address 127.11.192.252:36667)
0812 07:03:16.238947 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:16.238953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:16.238994 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:03:16.239059 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:16.239065 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:16.239136 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:16.239812 (+ 676us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:16.240101 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.241374 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.243937 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.244905 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.245064 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:16.245130 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:16.245439 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 07:03:16.245563 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":770,"thread_start_us":785,"threads_started":1}
I20260812 07:03:16.245814 24188 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 998954c32f6b45cba1277aacf2dc5196 in term 1.
W20260812 07:03:16.245976 24149 leader_election.cc:334] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f729c2efda9346f2a36c4ba13c5c56f9 (127.11.192.254:45769): Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
I20260812 07:03:16.246074 25505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.237754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:36667 (local address 127.0.0.1:39018)
0812 07:03:16.238222 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.238240 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:16.238411 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:16.239308 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:16.239314 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:16.239342 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:16.239668 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:16.239679 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.241553 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.241560 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:16.243742 (+ 2182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:16.243761 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.245100 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.245108 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:16.245290 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:16.245315 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:03:16.245409 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 07:03:16.245561 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":4,"thread_start_us":6111,"threads_started":1}
I20260812 07:03:16.246500 24472 leader_election.cc:302] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8d82aaf8d7ef451dabbed9397999081c, 998954c32f6b45cba1277aacf2dc5196; no voters: f729c2efda9346f2a36c4ba13c5c56f9
I20260812 07:03:16.246831 25498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 07:03:16.246934 25498 raft_consensus.cc:482] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f729c2efda9346f2a36c4ba13c5c56f9)
I20260812 07:03:16.247035 25498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [term 1 FOLLOWER]: Advancing to term 2
I20260812 07:03:16.247303 24521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8d82aaf8d7ef451dabbed9397999081c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "998954c32f6b45cba1277aacf2dc5196" is_pre_election: true
I20260812 07:03:16.249398 25498 raft_consensus.cc:504] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: false } }
I20260812 07:03:16.249854 24521 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d82aaf8d7ef451dabbed9397999081c in current term 2: Already voted for candidate 998954c32f6b45cba1277aacf2dc5196 in this term.
I20260812 07:03:16.250094 25498 leader_election.cc:288] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [CANDIDATE]: Term 2 election: Requested vote from peers f729c2efda9346f2a36c4ba13c5c56f9 (127.11.192.254:45769), 8d82aaf8d7ef451dabbed9397999081c (127.11.192.253:41945)
I20260812 07:03:16.250546 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.250144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.0.0.1:44664)
0812 07:03:16.250328 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.250413 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260812 07:03:16.250595 24188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "998954c32f6b45cba1277aacf2dc5196" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8d82aaf8d7ef451dabbed9397999081c"
I20260812 07:03:16.250820 24188 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 1 FOLLOWER]: Advancing to term 2
I20260812 07:03:16.251487 24154 leader_election.cc:302] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d82aaf8d7ef451dabbed9397999081c; no voters: 998954c32f6b45cba1277aacf2dc5196, f729c2efda9346f2a36c4ba13c5c56f9
I20260812 07:03:16.252375 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.251126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.0.0.1:44666)
0812 07:03:16.252150 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.252241 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":904}
W20260812 07:03:16.252557 24466 leader_election.cc:334] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f729c2efda9346f2a36c4ba13c5c56f9 (127.11.192.254:45769): Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
I20260812 07:03:16.253471 24188 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998954c32f6b45cba1277aacf2dc5196 in term 2.
I20260812 07:03:16.253914 24472 leader_election.cc:302] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8d82aaf8d7ef451dabbed9397999081c, 998954c32f6b45cba1277aacf2dc5196; no voters: f729c2efda9346f2a36c4ba13c5c56f9
I20260812 07:03:16.254314 25498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 07:03:16.254921 25498 raft_consensus.cc:686] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [term 2 LEADER]: Becoming Leader. State: Replica: 998954c32f6b45cba1277aacf2dc5196, State: Running, Role: LEADER
I20260812 07:03:16.255060 25502 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 07:03:16.255303 25498 consensus_queue.cc:237] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: false } }
I20260812 07:03:16.257390 25517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 998954c32f6b45cba1277aacf2dc5196. Latest consensus state: current_term: 2 leader_uuid: "998954c32f6b45cba1277aacf2dc5196" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: false } } }
I20260812 07:03:16.257737 25517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:16.258373 25522 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:03:16.261765 25522 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:03:16.262521 25522 catalog_manager.cc:1259] Loaded cluster ID: 13611160523f4246a5374c1ee4313dbb
I20260812 07:03:16.262591 25522 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:03:16.263619 25522 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:03:16.264443 25522 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196: Loaded TSK: 0
I20260812 07:03:16.265236 25522 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:03:16.309628 24501 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:03:16.316782 24501 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:03:16.320607 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.320211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.0.0.1:44694)
0812 07:03:16.320349 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.320434 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 07:03:16.320745 24188 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [term 2 FOLLOWER]: Refusing update from remote peer 998954c32f6b45cba1277aacf2dc5196: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260812 07:03:16.321484 25517 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260812 07:03:16.321620 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.321251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.0.0.1:44706)
0812 07:03:16.321418 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.321495 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 07:03:16.321882 24466 consensus_peers.cc:487] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 -> Peer f729c2efda9346f2a36c4ba13c5c56f9 (127.11.192.254:45769): Couldn't send request to peer f729c2efda9346f2a36c4ba13c5c56f9. Status: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 07:03:16.323794 25502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 998954c32f6b45cba1277aacf2dc5196. Latest consensus state: current_term: 2 leader_uuid: "998954c32f6b45cba1277aacf2dc5196" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: false } } }
I20260812 07:03:16.324034 25502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:03:16.325565 25517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "998954c32f6b45cba1277aacf2dc5196" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: false } } }
I20260812 07:03:16.325838 25517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:16.326335 25502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "998954c32f6b45cba1277aacf2dc5196" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f729c2efda9346f2a36c4ba13c5c56f9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45769 } } peers { permanent_uuid: "8d82aaf8d7ef451dabbed9397999081c" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41945 } attrs { promote: false } } peers { permanent_uuid: "998954c32f6b45cba1277aacf2dc5196" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 36667 } attrs { promote: false } } }
I20260812 07:03:16.326629 25502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d82aaf8d7ef451dabbed9397999081c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:03:16.327095 25525 mvcc.cc:205] Tried to move back new op lower bound from 7317578532123435008 to 7317578531864973312. Current Snapshot: MvccSnapshot[applied={T|T < 7317578532123435008 or (T in {7317578532123435008})}]
I20260812 07:03:16.344398 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.333613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:34145 (local address 127.0.0.1:56212)
0812 07:03:16.333843 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.333859 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:16.333950 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:16.335203 (+ 1253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:16.335209 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:16.335238 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:16.335597 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:16.335608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.338590 (+ 2982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.338597 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:16.340836 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:16.340853 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.343588 (+ 2735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.343595 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:16.343807 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:16.343848 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 07:03:16.343916 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 07:03:16.344067 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125}
I20260812 07:03:16.346509 25536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.334000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56212 (local address 127.11.192.194:34145)
0812 07:03:16.334639 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:16.334645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:16.334664 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:03:16.334734 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:16.334739 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:16.335017 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:16.335744 (+ 727us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:16.335911 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.338419 (+ 2508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.342060 (+ 3641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.343328 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.344919 (+ 1591us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:16.345000 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:16.346043 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0812 07:03:16.346192 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":407,"threads_started":1}
I20260812 07:03:16.348783 24963 tablet_service.cc:1467] Processing CreateTablet for tablet 313d6f2f3a0b4f388fd0d37cce164972 (DEFAULT_TABLE table=test-table [id=516fb4641f234be4943c38a9a0e7b797]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:03:16.349229 24963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 313d6f2f3a0b4f388fd0d37cce164972. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:16.352295 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.334174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:42031 (local address 127.0.0.1:58250)
0812 07:03:16.344618 (+ 10444us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.344634 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:16.344721 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:16.346175 (+ 1454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:16.346179 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:16.346224 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:16.346378 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:16.346388 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.347750 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.347756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:16.349600 (+ 1844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:16.349615 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.351579 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.351585 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:16.351760 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:16.351783 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 07:03:16.351861 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 07:03:16.351985 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10361,"thread_start_us":13728,"threads_started":1}
I20260812 07:03:16.352357 25539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.334313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58250 (local address 127.11.192.193:42031)
0812 07:03:16.345734 (+ 11421us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:16.345740 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:16.345760 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:03:16.345831 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:16.345836 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:16.345915 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:16.346549 (+ 634us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:16.346707 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.347624 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.349778 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.351418 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.351584 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:16.351663 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:16.351899 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 07:03:16.352037 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11295,"thread_start_us":11687,"threads_started":1}
I20260812 07:03:16.352775 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.333212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:41935 (local address 127.0.0.1:39924)
0812 07:03:16.333359 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.333373 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:16.333462 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:16.347007 (+ 13545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:16.347012 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:16.347039 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:16.347203 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:16.347214 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.348618 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.348624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:16.350399 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:16.350411 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.351987 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.351993 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:16.352255 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:16.352294 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 07:03:16.352398 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 07:03:16.352528 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 07:03:16.353235 24714 tablet_service.cc:1467] Processing CreateTablet for tablet 313d6f2f3a0b4f388fd0d37cce164972 (DEFAULT_TABLE table=test-table [id=516fb4641f234be4943c38a9a0e7b797]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:03:16.353663 24714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 313d6f2f3a0b4f388fd0d37cce164972. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:16.355434 25540 tablet_bootstrap.cc:492] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30: Bootstrap starting.
I20260812 07:03:16.357095 25540 tablet_bootstrap.cc:654] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:16.357616 25535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.333690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39924 (local address 127.11.192.195:41935)
0812 07:03:16.346626 (+ 12936us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:16.346632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:16.346655 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:03:16.346734 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:16.346739 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:16.346818 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:16.346965 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:16.347335 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.348461 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.350561 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.351790 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.357069 (+ 5279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:16.357143 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:16.357195 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 07:03:16.357321 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12640,"thread_start_us":12774,"threads_started":1}
I20260812 07:03:16.358367 25249 tablet_service.cc:1467] Processing CreateTablet for tablet 313d6f2f3a0b4f388fd0d37cce164972 (DEFAULT_TABLE table=test-table [id=516fb4641f234be4943c38a9a0e7b797]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:03:16.358805 25249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 313d6f2f3a0b4f388fd0d37cce164972. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:16.359337 25540 tablet_bootstrap.cc:492] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30: No bootstrap required, opened a new log
I20260812 07:03:16.359465 25540 ts_tablet_manager.cc:1397] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260812 07:03:16.359474 25545 tablet_bootstrap.cc:492] T 313d6f2f3a0b4f388fd0d37cce164972 P c58ec4c6df7d4bed947f921c427777c7: Bootstrap starting.
I20260812 07:03:16.360250 25540 raft_consensus.cc:348] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "476ffa1a46d444158f741ef6f66efada" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41935 } } peers { permanent_uuid: "fcfbae397ccb4992985a1147955cba30" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34145 } } peers { permanent_uuid: "c58ec4c6df7d4bed947f921c427777c7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 42031 } }
I20260812 07:03:16.360492 25540 raft_consensus.cc:374] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:16.360574 25540 raft_consensus.cc:729] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcfbae397ccb4992985a1147955cba30, State: Initialized, Role: FOLLOWER
I20260812 07:03:16.360811 25540 consensus_queue.cc:260] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "476ffa1a46d444158f741ef6f66efada" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41935 } } peers { permanent_uuid: "fcfbae397ccb4992985a1147955cba30" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34145 } } peers { permanent_uuid: "c58ec4c6df7d4bed947f921c427777c7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 42031 } }
I20260812 07:03:16.361212 25545 tablet_bootstrap.cc:654] T 313d6f2f3a0b4f388fd0d37cce164972 P c58ec4c6df7d4bed947f921c427777c7: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:16.361609 25540 ts_tablet_manager.cc:1428] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 07:03:16.363401 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.362447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.11.192.194:46735)
0812 07:03:16.363012 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.363172 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"threads_started":1}
I20260812 07:03:16.364917 25553 tablet_bootstrap.cc:492] T 313d6f2f3a0b4f388fd0d37cce164972 P 476ffa1a46d444158f741ef6f66efada: Bootstrap starting.
I20260812 07:03:16.366449 25553 tablet_bootstrap.cc:654] T 313d6f2f3a0b4f388fd0d37cce164972 P 476ffa1a46d444158f741ef6f66efada: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:16.366875 25087 heartbeater.cc:503] Master 127.11.192.252:36667 was elected leader, sending a full tablet report...
I20260812 07:03:16.367543 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.367127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.11.192.194:45141)
0812 07:03:16.367324 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.367404 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260812 07:03:16.370172 25553 tablet_bootstrap.cc:492] T 313d6f2f3a0b4f388fd0d37cce164972 P 476ffa1a46d444158f741ef6f66efada: No bootstrap required, opened a new log
I20260812 07:03:16.370287 25553 ts_tablet_manager.cc:1397] T 313d6f2f3a0b4f388fd0d37cce164972 P 476ffa1a46d444158f741ef6f66efada: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.003s
I20260812 07:03:16.370711 25545 tablet_bootstrap.cc:492] T 313d6f2f3a0b4f388fd0d37cce164972 P c58ec4c6df7d4bed947f921c427777c7: No bootstrap required, opened a new log
I20260812 07:03:16.370838 25545 ts_tablet_manager.cc:1397] T 313d6f2f3a0b4f388fd0d37cce164972 P c58ec4c6df7d4bed947f921c427777c7: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.004s
I20260812 07:03:16.370887 25553 raft_consensus.cc:348] T 313d6f2f3a0b4f388fd0d37cce164972 P 476ffa1a46d444158f741ef6f66efada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "476ffa1a46d444158f741ef6f66efada" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41935 } } peers { permanent_uuid: "fcfbae397ccb4992985a1147955cba30" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34145 } } peers { permanent_uuid: "c58ec4c6df7d4bed947f921c427777c7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 42031 } }
I20260812 07:03:16.371074 25553 raft_consensus.cc:374] T 313d6f2f3a0b4f388fd0d37cce164972 P 476ffa1a46d444158f741ef6f66efada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:16.371167 25553 raft_consensus.cc:729] T 313d6f2f3a0b4f388fd0d37cce164972 P 476ffa1a46d444158f741ef6f66efada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 476ffa1a46d444158f741ef6f66efada, State: Initialized, Role: FOLLOWER
I20260812 07:03:16.371366 25553 consensus_queue.cc:260] T 313d6f2f3a0b4f388fd0d37cce164972 P 476ffa1a46d444158f741ef6f66efada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "476ffa1a46d444158f741ef6f66efada" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41935 } } peers { permanent_uuid: "fcfbae397ccb4992985a1147955cba30" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34145 } } peers { permanent_uuid: "c58ec4c6df7d4bed947f921c427777c7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 42031 } }
I20260812 07:03:16.371516 25545 raft_consensus.cc:348] T 313d6f2f3a0b4f388fd0d37cce164972 P c58ec4c6df7d4bed947f921c427777c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "476ffa1a46d444158f741ef6f66efada" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41935 } } peers { permanent_uuid: "fcfbae397ccb4992985a1147955cba30" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34145 } } peers { permanent_uuid: "c58ec4c6df7d4bed947f921c427777c7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 42031 } }
I20260812 07:03:16.371770 25545 raft_consensus.cc:374] T 313d6f2f3a0b4f388fd0d37cce164972 P c58ec4c6df7d4bed947f921c427777c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:16.371876 25545 raft_consensus.cc:729] T 313d6f2f3a0b4f388fd0d37cce164972 P c58ec4c6df7d4bed947f921c427777c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c58ec4c6df7d4bed947f921c427777c7, State: Initialized, Role: FOLLOWER
I20260812 07:03:16.372272 25553 ts_tablet_manager.cc:1428] T 313d6f2f3a0b4f388fd0d37cce164972 P 476ffa1a46d444158f741ef6f66efada: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 07:03:16.372241 25545 consensus_queue.cc:260] T 313d6f2f3a0b4f388fd0d37cce164972 P c58ec4c6df7d4bed947f921c427777c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "476ffa1a46d444158f741ef6f66efada" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41935 } } peers { permanent_uuid: "fcfbae397ccb4992985a1147955cba30" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34145 } } peers { permanent_uuid: "c58ec4c6df7d4bed947f921c427777c7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 42031 } }
I20260812 07:03:16.374230 25364 heartbeater.cc:503] Master 127.11.192.252:36667 was elected leader, sending a full tablet report...
I20260812 07:03:16.375348 25559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.372912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.11.192.195:51165)
0812 07:03:16.374941 (+ 2029us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.375127 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1915,"thread_start_us":1798,"threads_started":1}
I20260812 07:03:16.376623 25559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.376015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.11.192.195:33921)
0812 07:03:16.376458 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.376527 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365}
I20260812 07:03:16.380663 25545 ts_tablet_manager.cc:1428] T 313d6f2f3a0b4f388fd0d37cce164972 P c58ec4c6df7d4bed947f921c427777c7: Time spent starting tablet: real 0.010s user 0.007s sys 0.001s
I20260812 07:03:16.381918 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.380667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.11.192.193:51301)
0812 07:03:16.381504 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.381663 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":546,"threads_started":1}
I20260812 07:03:16.382411 24827 heartbeater.cc:503] Master 127.11.192.252:36667 was elected leader, sending a full tablet report...
I20260812 07:03:16.382925 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.382563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.11.192.193:42527)
0812 07:03:16.382718 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.382800 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 07:03:16.484388 25546 raft_consensus.cc:482] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:03:16.484603 25546 raft_consensus.cc:504] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "476ffa1a46d444158f741ef6f66efada" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41935 } } peers { permanent_uuid: "fcfbae397ccb4992985a1147955cba30" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34145 } } peers { permanent_uuid: "c58ec4c6df7d4bed947f921c427777c7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 42031 } }
I20260812 07:03:16.486151 25546 leader_election.cc:288] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 476ffa1a46d444158f741ef6f66efada (127.11.192.195:41935), c58ec4c6df7d4bed947f921c427777c7 (127.11.192.193:42031)
I20260812 07:03:16.494230 25539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.486616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:54163 (local address 127.11.192.193:42031)
0812 07:03:16.486820 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:16.486826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:16.487164 (+ 338us) server_negotiation.cc:407] Connection header received
0812 07:03:16.487410 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:16.487416 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:16.487491 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:16.487577 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:16.488152 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.489228 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.492000 (+ 2772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.493203 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.493354 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:16.493425 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:16.493849 (+ 424us) server_negotiation.cc:299] Negotiation successful
0812 07:03:16.493984 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:03:16.494223 25535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.487182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:59765 (local address 127.11.192.195:41935)
0812 07:03:16.487394 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:16.487398 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:16.487412 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:03:16.487468 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:16.487473 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:16.487529 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:16.487608 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:16.488300 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.489315 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.491984 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.493203 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.493350 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:16.493425 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:16.493849 (+ 424us) server_negotiation.cc:299] Negotiation successful
0812 07:03:16.493977 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:03:16.494299 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.486362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:41935 (local address 127.11.192.194:59765)
0812 07:03:16.486692 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.486717 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:16.486845 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:16.487750 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:16.487758 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:16.487808 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:16.488155 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:16.488166 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.489432 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.489436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:16.491842 (+ 2406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:16.491854 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.493471 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.493477 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:16.493677 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:16.493738 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 07:03:16.493822 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 07:03:16.493978 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":135}
I20260812 07:03:16.494375 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.486408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:42031 (local address 127.11.192.194:54163)
0812 07:03:16.487064 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.487078 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:16.487194 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:16.487609 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:16.487614 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:16.487648 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:16.487989 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:16.487998 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.489361 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.489366 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:16.491838 (+ 2472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:16.491854 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.493444 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.493454 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:16.493682 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:16.493727 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 07:03:16.493822 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 07:03:16.493975 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":205,"thread_start_us":415,"threads_started":1}
I20260812 07:03:16.495107 24747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "313d6f2f3a0b4f388fd0d37cce164972" candidate_uuid: "fcfbae397ccb4992985a1147955cba30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c58ec4c6df7d4bed947f921c427777c7" is_pre_election: true
I20260812 07:03:16.495378 24747 raft_consensus.cc:2454] T 313d6f2f3a0b4f388fd0d37cce164972 P c58ec4c6df7d4bed947f921c427777c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcfbae397ccb4992985a1147955cba30 in term 0.
I20260812 07:03:16.495261 25279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "313d6f2f3a0b4f388fd0d37cce164972" candidate_uuid: "fcfbae397ccb4992985a1147955cba30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "476ffa1a46d444158f741ef6f66efada" is_pre_election: true
I20260812 07:03:16.495720 25279 raft_consensus.cc:2454] T 313d6f2f3a0b4f388fd0d37cce164972 P 476ffa1a46d444158f741ef6f66efada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcfbae397ccb4992985a1147955cba30 in term 0.
I20260812 07:03:16.495898 24895 leader_election.cc:302] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c58ec4c6df7d4bed947f921c427777c7, fcfbae397ccb4992985a1147955cba30; no voters:
I20260812 07:03:16.496320 25546 raft_consensus.cc:2781] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:03:16.496434 25546 raft_consensus.cc:482] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:03:16.496539 25546 raft_consensus.cc:3037] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:16.498634 25546 raft_consensus.cc:504] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "476ffa1a46d444158f741ef6f66efada" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41935 } } peers { permanent_uuid: "fcfbae397ccb4992985a1147955cba30" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34145 } } peers { permanent_uuid: "c58ec4c6df7d4bed947f921c427777c7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 42031 } }
I20260812 07:03:16.499154 25546 leader_election.cc:288] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30 [CANDIDATE]: Term 1 election: Requested vote from peers 476ffa1a46d444158f741ef6f66efada (127.11.192.195:41935), c58ec4c6df7d4bed947f921c427777c7 (127.11.192.193:42031)
I20260812 07:03:16.499583 25279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "313d6f2f3a0b4f388fd0d37cce164972" candidate_uuid: "fcfbae397ccb4992985a1147955cba30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "476ffa1a46d444158f741ef6f66efada"
I20260812 07:03:16.499610 24747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "313d6f2f3a0b4f388fd0d37cce164972" candidate_uuid: "fcfbae397ccb4992985a1147955cba30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c58ec4c6df7d4bed947f921c427777c7"
I20260812 07:03:16.499742 25279 raft_consensus.cc:3037] T 313d6f2f3a0b4f388fd0d37cce164972 P 476ffa1a46d444158f741ef6f66efada [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:16.499785 24747 raft_consensus.cc:3037] T 313d6f2f3a0b4f388fd0d37cce164972 P c58ec4c6df7d4bed947f921c427777c7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:16.501788 24747 raft_consensus.cc:2454] T 313d6f2f3a0b4f388fd0d37cce164972 P c58ec4c6df7d4bed947f921c427777c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcfbae397ccb4992985a1147955cba30 in term 1.
I20260812 07:03:16.501829 25279 raft_consensus.cc:2454] T 313d6f2f3a0b4f388fd0d37cce164972 P 476ffa1a46d444158f741ef6f66efada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcfbae397ccb4992985a1147955cba30 in term 1.
I20260812 07:03:16.502156 24895 leader_election.cc:302] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c58ec4c6df7d4bed947f921c427777c7, fcfbae397ccb4992985a1147955cba30; no voters:
I20260812 07:03:16.502427 25546 raft_consensus.cc:2781] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:03:16.502842 25546 raft_consensus.cc:686] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30 [term 1 LEADER]: Becoming Leader. State: Replica: fcfbae397ccb4992985a1147955cba30, State: Running, Role: LEADER
I20260812 07:03:16.503237 25546 consensus_queue.cc:237] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "476ffa1a46d444158f741ef6f66efada" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41935 } } peers { permanent_uuid: "fcfbae397ccb4992985a1147955cba30" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34145 } } peers { permanent_uuid: "c58ec4c6df7d4bed947f921c427777c7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 42031 } }
I20260812 07:03:16.503742 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.503332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.11.192.194:39131)
0812 07:03:16.503514 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.503604 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 07:03:16.506458 24502 catalog_manager.cc:5495] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30 reported cstate change: term changed from 0 to 1, leader changed from <none> to fcfbae397ccb4992985a1147955cba30 (127.11.192.194). New cstate: current_term: 1 leader_uuid: "fcfbae397ccb4992985a1147955cba30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "476ffa1a46d444158f741ef6f66efada" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcfbae397ccb4992985a1147955cba30" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c58ec4c6df7d4bed947f921c427777c7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 42031 } health_report { overall_health: UNKNOWN } } }
I20260812 07:03:16.530783 25536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.526363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56220 (local address 127.11.192.194:34145)
0812 07:03:16.526519 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:16.526523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:16.526539 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:03:16.526715 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:16.526718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:16.526787 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:16.526899 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:03:16.527167 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.527842 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.529068 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.529449 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.529536 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:16.529838 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:03:16.530349 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:03:16.530450 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:16.530490 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 07:03:16.530547 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 07:03:16.534409 24747 raft_consensus.cc:1261] T 313d6f2f3a0b4f388fd0d37cce164972 P c58ec4c6df7d4bed947f921c427777c7 [term 1 FOLLOWER]: Refusing update from remote peer fcfbae397ccb4992985a1147955cba30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:03:16.534727 25279 raft_consensus.cc:1261] T 313d6f2f3a0b4f388fd0d37cce164972 P 476ffa1a46d444158f741ef6f66efada [term 1 FOLLOWER]: Refusing update from remote peer fcfbae397ccb4992985a1147955cba30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:03:16.535094 25563 consensus_queue.cc:1035] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c58ec4c6df7d4bed947f921c427777c7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 42031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:03:16.535445 25546 consensus_queue.cc:1035] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "476ffa1a46d444158f741ef6f66efada" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:03:16.537896 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.537319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.11.192.193:52499)
0812 07:03:16.537573 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.537719 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260812 07:03:16.538486 25559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.538050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.11.192.195:56367)
0812 07:03:16.538206 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.538317 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 07:03:16.548601 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.548189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.11.192.193:54053)
0812 07:03:16.548329 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.548418 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 07:03:16.550438 25559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.549951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.11.192.195:43557)
0812 07:03:16.550206 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.550285 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260812 07:03:16.552671 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.552229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.11.192.194:45623)
0812 07:03:16.552390 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.552479 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 07:03:16.554345 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.553407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.11.192.194:39259)
0812 07:03:16.554128 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.554213 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":615}
I20260812 07:03:16.562513 24502 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:03:16.562723 24502 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:03:16.567564 24502 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 998954c32f6b45cba1277aacf2dc5196: Sending DeleteTablet for 3 replicas of tablet 313d6f2f3a0b4f388fd0d37cce164972
I20260812 07:03:16.568517 25249 tablet_service.cc:1514] Processing DeleteTablet for tablet 313d6f2f3a0b4f388fd0d37cce164972 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:03:16 UTC) from {username='oryx', principal='oryx/127.11.192.252@KRBTEST.COM'} at 127.0.0.1:39924
I20260812 07:03:16.568990 24963 tablet_service.cc:1514] Processing DeleteTablet for tablet 313d6f2f3a0b4f388fd0d37cce164972 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:03:16 UTC) from {username='oryx', principal='oryx/127.11.192.252@KRBTEST.COM'} at 127.0.0.1:56212
I20260812 07:03:16.569108 25586 tablet_replica.cc:330] T 313d6f2f3a0b4f388fd0d37cce164972 P 476ffa1a46d444158f741ef6f66efada: stopping tablet replica
I20260812 07:03:16.569278 25586 raft_consensus.cc:2229] T 313d6f2f3a0b4f388fd0d37cce164972 P 476ffa1a46d444158f741ef6f66efada [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:03:16.569380 24714 tablet_service.cc:1514] Processing DeleteTablet for tablet 313d6f2f3a0b4f388fd0d37cce164972 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:03:16 UTC) from {username='oryx', principal='oryx/127.11.192.252@KRBTEST.COM'} at 127.0.0.1:58250
I20260812 07:03:16.569571 25586 raft_consensus.cc:2258] T 313d6f2f3a0b4f388fd0d37cce164972 P 476ffa1a46d444158f741ef6f66efada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:03:16.569996 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 24461
I20260812 07:03:16.572790 25586 ts_tablet_manager.cc:1905] T 313d6f2f3a0b4f388fd0d37cce164972 P 476ffa1a46d444158f741ef6f66efada: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:03:16.579229 25586 ts_tablet_manager.cc:1918] T 313d6f2f3a0b4f388fd0d37cce164972 P 476ffa1a46d444158f741ef6f66efada: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:03:16.579423 25586 log.cc:1196] T 313d6f2f3a0b4f388fd0d37cce164972 P 476ffa1a46d444158f741ef6f66efada: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/313d6f2f3a0b4f388fd0d37cce164972
I20260812 07:03:16.580024 25586 ts_tablet_manager.cc:1939] T 313d6f2f3a0b4f388fd0d37cce164972 P 476ffa1a46d444158f741ef6f66efada: Deleting consensus metadata
I20260812 07:03:16.581291 24466 catalog_manager.cc:4841] TS 476ffa1a46d444158f741ef6f66efada (127.11.192.195:41935): tablet 313d6f2f3a0b4f388fd0d37cce164972 (table test-table [id=516fb4641f234be4943c38a9a0e7b797]) successfully deleted
I20260812 07:03:16.587049 25539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.571468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58254 (local address 127.11.192.193:42031)
0812 07:03:16.571636 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:16.571643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:16.572049 (+ 406us) server_negotiation.cc:407] Connection header received
0812 07:03:16.572338 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:16.572343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:16.572434 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:16.572505 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:16.574379 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.575530 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.577222 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.577831 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.577924 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:16.581460 (+ 3536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:16.581479 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:16.581496 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:16.581536 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:16.584472 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:16.585064 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:16.585071 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:16.585073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:16.585195 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:16.585553 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:16.585557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:16.585559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:16.585900 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:16.586189 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:16.586467 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 07:03:16.586685 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:03:16.587625 24819 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:16.591279 25589 tablet_replica.cc:330] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30: stopping tablet replica
I20260812 07:03:16.591509 25589 raft_consensus.cc:2229] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:03:16.592260 25589 raft_consensus.cc:2258] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:03:16.593339 25589 ts_tablet_manager.cc:1905] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:03:16.600298 25589 ts_tablet_manager.cc:1918] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:03:16.600448 25589 log.cc:1196] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/313d6f2f3a0b4f388fd0d37cce164972
I20260812 07:03:16.600950 25589 ts_tablet_manager.cc:1939] T 313d6f2f3a0b4f388fd0d37cce164972 P fcfbae397ccb4992985a1147955cba30: Deleting consensus metadata
I20260812 07:03:16.602101 24472 catalog_manager.cc:4841] TS fcfbae397ccb4992985a1147955cba30 (127.11.192.194:34145): tablet 313d6f2f3a0b4f388fd0d37cce164972 (table test-table [id=516fb4641f234be4943c38a9a0e7b797]) successfully deleted
I20260812 07:03:16.679804 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 24461
I20260812 07:03:16.708326 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 24853
I20260812 07:03:16.708993 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.708404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:42031 (local address 127.0.0.1:58260)
0812 07:03:16.708663 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.708763 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:42031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260812 07:03:16.709375 24472 catalog_manager.cc:4584] TS c58ec4c6df7d4bed947f921c427777c7 (127.11.192.193:42031): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 313d6f2f3a0b4f388fd0d37cce164972: Network error: Client connection negotiation failed: client connection to 127.11.192.193:42031: connect: Connection refused (error 111)
I20260812 07:03:16.723500 25536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.709064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56236 (local address 127.11.192.194:34145)
0812 07:03:16.709455 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:16.709461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:16.709484 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:03:16.709552 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:16.709559 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:16.709640 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:16.709743 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:16.710960 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.712361 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.713768 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.714434 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.714572 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:16.716993 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:16.717012 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:16.717034 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:16.717082 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:16.720854 (+ 3772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:16.721468 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:16.721478 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:16.721482 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:16.721655 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:16.721961 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:16.721966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:16.721978 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:16.722419 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:16.722746 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:16.722884 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 07:03:16.723114 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258}
I20260812 07:03:16.724193 25078 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:16.769913 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.769513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:42031 (local address 127.0.0.1:58276)
0812 07:03:16.769689 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.769768 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:42031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 07:03:16.770892 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.770544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:42031 (local address 127.0.0.1:58278)
0812 07:03:16.770689 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.770763 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:42031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 07:03:16.837571 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 24853
I20260812 07:03:16.841672 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.841235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:42031 (local address 127.0.0.1:58288)
0812 07:03:16.841434 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.841543 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:42031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 07:03:16.842691 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.842379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:42031 (local address 127.0.0.1:58302)
0812 07:03:16.842505 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.842577 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:42031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 07:03:16.861642 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 25114
I20260812 07:03:16.873824 25535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.862308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39932 (local address 127.11.192.195:41935)
0812 07:03:16.862464 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:16.862469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:16.862487 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:03:16.862631 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:16.862634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:16.862696 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:16.862772 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:16.864064 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.864813 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.865967 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:16.866380 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:16.866466 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:16.869079 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:16.869094 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:16.869111 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:16.869144 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:16.871637 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:16.872240 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:16.872247 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:16.872249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:16.872375 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:16.872644 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:16.872646 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:16.872648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:16.872994 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:16.873245 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:16.873441 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 07:03:16.873599 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:03:16.874436 25358 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:16.929111 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.928654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.0.0.1:44716)
0812 07:03:16.928828 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.928934 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 07:03:16.930131 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.929731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45769 (local address 127.0.0.1:44730)
0812 07:03:16.929890 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.929977 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:45769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 07:03:16.944440 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.944119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:42031 (local address 127.0.0.1:58306)
0812 07:03:16.944256 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.944334 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:42031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 07:03:16.945369 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:16.945066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:42031 (local address 127.0.0.1:58314)
0812 07:03:16.945183 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:16.945253 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:42031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 07:03:16.992439 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 25114
I20260812 07:03:17.020817 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 23915
I20260812 07:03:17.021852 24224 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:17.117116 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:17.116650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:42031 (local address 127.0.0.1:58318)
0812 07:03:17.116864 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:17.116958 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:42031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 07:03:17.118213 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:17.117849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:42031 (local address 127.0.0.1:58324)
0812 07:03:17.117998 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:17.118083 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:42031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 07:03:17.139285 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 23915
I20260812 07:03:17.164914 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 24110
I20260812 07:03:17.165997 24567 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:17.284564 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 24110
2026-08-12T07:03:17Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9774 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:03:17 dist-test-slave-lgr5 krb5kdc[25605](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:03:17 dist-test-slave-lgr5 krb5kdc[25605](info): set up 2 sockets
Aug 12 07:03:17 dist-test-slave-lgr5 krb5kdc[25605](info): commencing operation
krb5kdc: starting...
W20260812 07:03:18.522293 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.002s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:03:18 dist-test-slave-lgr5 krb5kdc[25605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:03:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:03:18Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.254.keytab.
Entry for principal oryx/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.254.keytab.
I20260812 07:03:18.730930 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:43629
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.254.keytab
--principal=oryx/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:40167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:43629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:19.043672 25634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:19.043923 25634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:19.044049 25634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:19.050868 25634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:03:19.050952 25634 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 07:03:19.051017 25634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:19.051100 25634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:19.051147 25634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:03:19.051221 25634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:03:19.060400 25634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:40167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:43629
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.254.keytab
--principal=oryx/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:43629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:19.061440 25634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:19.062572 25634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:19.073330 25645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:19.073549 25646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:19.073549 25650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:19.074220 25634 server_base.cc:1034] running on GCE node
I20260812 07:03:19.074903 25634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:19.076136 25634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:19.077353 25634 hybrid_clock.cc:648] HybridClock initialized: now 1786518199077287 us; error 42 us; skew 500 ppm
I20260812 07:03:19.077644 25634 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:19 dist-test-slave-lgr5 krb5kdc[25605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518199, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:19.080507 25634 init.cc:377] Logged in from keytab as oryx/127.11.192.254@KRBTEST.COM (short username oryx)
I20260812 07:03:19.081787 25634 webserver.cc:460] Webserver started at http://127.11.192.254:45973/ using document root <none> and password file <none>
I20260812 07:03:19.082408 25634 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:19.082515 25634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:19.082811 25634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:19.085582 25634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "67b1650a4f4a4768b5412d496251a883"
format_stamp: "Formatted at 2026-08-12 07:03:19 on dist-test-slave-lgr5"
server_key: "8ec0620c10d22f162c3483b06c17fdcc"
server_key_iv: "5779e707496335c3f2048732b117a0fb"
server_key_version: "encryptionkey@0"
I20260812 07:03:19.086325 25634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "67b1650a4f4a4768b5412d496251a883"
format_stamp: "Formatted at 2026-08-12 07:03:19 on dist-test-slave-lgr5"
server_key: "8ec0620c10d22f162c3483b06c17fdcc"
server_key_iv: "5779e707496335c3f2048732b117a0fb"
server_key_version: "encryptionkey@0"
I20260812 07:03:19.090905 25634 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:03:19.094108 25661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:19.095250 25634 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:03:19.095403 25634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "67b1650a4f4a4768b5412d496251a883"
format_stamp: "Formatted at 2026-08-12 07:03:19 on dist-test-slave-lgr5"
server_key: "8ec0620c10d22f162c3483b06c17fdcc"
server_key_iv: "5779e707496335c3f2048732b117a0fb"
server_key_version: "encryptionkey@0"
I20260812 07:03:19.095603 25634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:19.108443 25634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:19.111893 25634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:19.112164 25634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:19.131431 25634 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:43629
I20260812 07:03:19.131450 25775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:43629 every 8 connection(s)
I20260812 07:03:19.132694 25634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:03:19.136165 25776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:19.138679 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 25634
I20260812 07:03:19.138900 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:03:19.139690 12035 external_mini_cluster.cc:1428] Setting key a4ea48263af8053c061ea99a463dd7e6
I20260812 07:03:19.148044 25776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883: Bootstrap starting.
I20260812 07:03:19.151574 25776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:19.152833 25776 log.cc:824] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883: Log is configured to *not* fsync() on all Append() calls
Aug 12 07:03:19 dist-test-slave-lgr5 krb5kdc[25605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM
I20260812 07:03:19.155740 25776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883: No bootstrap required, opened a new log
I20260812 07:03:19.161834 25781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:19.141653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42422 (local address 127.11.192.254:43629)
0812 07:03:19.142440 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:19.142450 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:19.142490 (+ 40us) server_negotiation.cc:407] Connection header received
0812 07:03:19.144169 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:19.144192 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:19.144556 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:19.144867 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:19.146553 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:19.147766 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:19.149412 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:19.150082 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:19.150202 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:19.154946 (+ 4744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:19.154953 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:19.154967 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:19.155007 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:19.157961 (+ 2954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:19.158726 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:19.158732 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:19.158737 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:19.158899 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:19.159275 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:19.159279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:19.159280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:19.159776 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:19.160022 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:19.160497 (+ 475us) server_negotiation.cc:299] Negotiation successful
0812 07:03:19.160856 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":403,"threads_started":1}
I20260812 07:03:19.164567 25776 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b1650a4f4a4768b5412d496251a883" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43629 } }
I20260812 07:03:19.164988 25776 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:19.165077 25776 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67b1650a4f4a4768b5412d496251a883, State: Initialized, Role: FOLLOWER
I20260812 07:03:19.165541 25776 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b1650a4f4a4768b5412d496251a883" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43629 } }
I20260812 07:03:19.165736 25776 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:03:19.165827 25776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:03:19.165954 25776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:19.167966 25776 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b1650a4f4a4768b5412d496251a883" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43629 } }
I20260812 07:03:19.168485 25776 leader_election.cc:302] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67b1650a4f4a4768b5412d496251a883; no voters:
I20260812 07:03:19.169080 25776 leader_election.cc:288] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:03:19.169200 25788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:03:19.169836 25788 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883 [term 1 LEADER]: Becoming Leader. State: Replica: 67b1650a4f4a4768b5412d496251a883, State: Running, Role: LEADER
I20260812 07:03:19.170397 25788 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b1650a4f4a4768b5412d496251a883" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43629 } }
I20260812 07:03:19.170961 25776 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:03:19.173085 25789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67b1650a4f4a4768b5412d496251a883" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b1650a4f4a4768b5412d496251a883" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43629 } } }
I20260812 07:03:19.173475 25789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:19.174435 25790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67b1650a4f4a4768b5412d496251a883. Latest consensus state: current_term: 1 leader_uuid: "67b1650a4f4a4768b5412d496251a883" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b1650a4f4a4768b5412d496251a883" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43629 } } }
I20260812 07:03:19.174659 25790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:19.177083 25803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:03:19.181414 25803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:03:19.188822 25803 catalog_manager.cc:1347] Generated new cluster ID: fef5a6c0d0914b3bba5e9e1c4abfc7c5
I20260812 07:03:19.188920 25803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:03:19.240810 25803 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:03:19.242616 25803 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:03:19.253144 25803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883: Generated new TSK 0
I20260812 07:03:19.254072 25803 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.193.keytab.
Entry for principal oryx/127.11.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.193.keytab.
I20260812 07:03:19.315133 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43629
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.193.keytab
--principal=oryx/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:40167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:19.629161 25822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:19.629403 25822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:19.629514 25822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:19.636380 25822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:19.636477 25822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:19.636564 25822 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:03:19.636641 25822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:03:19.645730 25822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:40167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.193.keytab
--principal=oryx/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.192.254:43629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:19.646809 25822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:19.647993 25822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:19.660055 25829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:19.659991 25833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:19.660017 25831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:19.660811 25822 server_base.cc:1034] running on GCE node
I20260812 07:03:19.661401 25822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:19.662213 25822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:19.663424 25822 hybrid_clock.cc:648] HybridClock initialized: now 1786518199663348 us; error 55 us; skew 500 ppm
I20260812 07:03:19.663682 25822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:19 dist-test-slave-lgr5 krb5kdc[25605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518199, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:19.666545 25822 init.cc:377] Logged in from keytab as oryx/127.11.192.193@KRBTEST.COM (short username oryx)
I20260812 07:03:19.667802 25822 webserver.cc:460] Webserver started at http://127.11.192.193:41251/ using document root <none> and password file <none>
I20260812 07:03:19.668462 25822 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:19.668565 25822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:19.668911 25822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:19.671727 25822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "e2b70782d5814ce8a9beb0f8712762c9"
format_stamp: "Formatted at 2026-08-12 07:03:19 on dist-test-slave-lgr5"
server_key: "a1299901cd4deb7cb711220917fb49b6"
server_key_iv: "31fa2e850ab66b6036e7f66efaf50522"
server_key_version: "encryptionkey@0"
I20260812 07:03:19.672508 25822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "e2b70782d5814ce8a9beb0f8712762c9"
format_stamp: "Formatted at 2026-08-12 07:03:19 on dist-test-slave-lgr5"
server_key: "a1299901cd4deb7cb711220917fb49b6"
server_key_iv: "31fa2e850ab66b6036e7f66efaf50522"
server_key_version: "encryptionkey@0"
I20260812 07:03:19.677424 25822 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 07:03:19.680552 25845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:19.681725 25822 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260812 07:03:19.681864 25822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "e2b70782d5814ce8a9beb0f8712762c9"
format_stamp: "Formatted at 2026-08-12 07:03:19 on dist-test-slave-lgr5"
server_key: "a1299901cd4deb7cb711220917fb49b6"
server_key_iv: "31fa2e850ab66b6036e7f66efaf50522"
server_key_version: "encryptionkey@0"
I20260812 07:03:19.682065 25822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:19.700440 25822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:19.703902 25822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:19.704170 25822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:19.706584 25822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:19.706682 25822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:19.706810 25822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:19.706880 25822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:03:19 dist-test-slave-lgr5 krb5kdc[25605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518199, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM
I20260812 07:03:19.728772 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:19.707196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:49869 (local address 127.11.192.254:43629)
0812 07:03:19.707747 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:19.707751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:19.708767 (+ 1016us) server_negotiation.cc:407] Connection header received
0812 07:03:19.710665 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:19.710670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:19.710749 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:19.710862 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:19.713142 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:19.714103 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:19.716896 (+ 2793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:19.717377 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:19.717504 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:19.722075 (+ 4571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:19.722079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:19.722083 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:19.722115 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:19.725270 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:19.726104 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:19.726110 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:19.726113 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:19.726276 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:19.726805 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:19.726811 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:19.726814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:19.727183 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:19.727328 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:19.728188 (+ 860us) server_negotiation.cc:299] Negotiation successful
0812 07:03:19.728408 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":353,"threads_started":1}
I20260812 07:03:19.729642 25862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:19.707806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43629 (local address 127.11.192.193:49869)
0812 07:03:19.708621 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:19.708655 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:19.710399 (+ 1744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:19.711151 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:19.711164 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:19.711960 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:19.712939 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:19.712973 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:19.714280 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:19.714286 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:19.716736 (+ 2450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:19.716756 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:19.717816 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:19.717826 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:19.718116 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:19.718832 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:19.718866 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:19.721844 (+ 2978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:19.725414 (+ 3570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:19.725422 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:19.725437 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:19.725962 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:19.726407 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:19.726413 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:19.726416 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:19.726640 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:19.727414 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:19.727422 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:19.727793 (+ 371us) client_negotiation.cc:769] Sending connection context
0812 07:03:19.728045 (+ 252us) client_negotiation.cc:240] Negotiation successful
0812 07:03:19.728543 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":363,"threads_started":1}
I20260812 07:03:19.748183 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:19.734925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:49703 (local address 127.11.192.254:43629)
0812 07:03:19.735064 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:19.735068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:19.735659 (+ 591us) server_negotiation.cc:407] Connection header received
0812 07:03:19.735797 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:19.735802 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:19.735860 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:19.735934 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:19.737206 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:19.738140 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:19.740043 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:19.740586 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:19.740707 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:19.742758 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:19.742763 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:19.742766 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:19.742800 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:19.745671 (+ 2871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:19.746257 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:19.746263 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:19.746266 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:19.746421 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:19.746838 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:19.746841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:19.746843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:19.747104 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:19.747241 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:19.747736 (+ 495us) server_negotiation.cc:299] Negotiation successful
0812 07:03:19.747922 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 07:03:19.748538 25979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:19.735007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43629 (local address 127.11.192.193:49703)
0812 07:03:19.735565 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:19.735579 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:19.735697 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:19.735971 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:19.735976 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:19.736579 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:19.737056 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:19.737075 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:19.738307 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:19.738312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:19.739909 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:19.739925 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:19.740933 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:19.740942 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:19.741128 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:19.741680 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:19.741698 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:19.742541 (+ 843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:19.745783 (+ 3242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:19.745788 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:19.745792 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:19.746147 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:19.746555 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:19.746562 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:19.746566 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:19.746733 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:19.747209 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:19.747215 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:19.747431 (+ 216us) client_negotiation.cc:769] Sending connection context
0812 07:03:19.747621 (+ 190us) client_negotiation.cc:240] Negotiation successful
0812 07:03:19.747961 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":298,"threads_started":1}
I20260812 07:03:19.754252 25822 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:40691
I20260812 07:03:19.754297 26055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:40691 every 8 connection(s)
I20260812 07:03:19.755348 25822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:03:19.757120 26057 heartbeater.cc:347] Connected to a master server at 127.11.192.254:43629
I20260812 07:03:19.757323 26057 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:19.757889 26057 heartbeater.cc:511] Master 127.11.192.254:43629 requested a full tablet report, sending...
I20260812 07:03:19.759591 25695 ts_manager.cc:194] Registered new tserver with Master: e2b70782d5814ce8a9beb0f8712762c9 (127.11.192.193:40691)
I20260812 07:03:19.760957 25695 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:49869
I20260812 07:03:19.764487 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 25822
I20260812 07:03:19.764675 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:03:19.765265 12035 external_mini_cluster.cc:1428] Setting key 8b03b32be767c1569d3b08233dd1639c
WARNING: no policy specified for oryx/127.11.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.194.keytab.
Entry for principal oryx/127.11.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.194.keytab.
I20260812 07:03:19.812245 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43629
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.194.keytab
--principal=oryx/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:40167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:20.127192 26063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:20.127451 26063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:20.127581 26063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:20.136758 26063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:20.136888 26063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:20.137038 26063 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:03:20.137161 26063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:03:20.152004 26063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:40167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.194.keytab
--principal=oryx/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.192.254:43629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:20.153475 26063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:20.154852 26063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:20.167092 26074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:20.167160 26072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:20.167133 26071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:20.167815 26063 server_base.cc:1034] running on GCE node
I20260812 07:03:20.168433 26063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:20.169235 26063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:20.170424 26063 hybrid_clock.cc:648] HybridClock initialized: now 1786518200170356 us; error 49 us; skew 500 ppm
I20260812 07:03:20.170707 26063 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:20 dist-test-slave-lgr5 krb5kdc[25605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518200, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:20.173789 26063 init.cc:377] Logged in from keytab as oryx/127.11.192.194@KRBTEST.COM (short username oryx)
I20260812 07:03:20.175221 26063 webserver.cc:460] Webserver started at http://127.11.192.194:37257/ using document root <none> and password file <none>
I20260812 07:03:20.175843 26063 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:20.175941 26063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:20.176285 26063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:20.179081 26063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "162f4d9eeba348f9a9e103b57d8a85ae"
format_stamp: "Formatted at 2026-08-12 07:03:20 on dist-test-slave-lgr5"
server_key: "1153dc71f79936ca3d3ee8178dd5316f"
server_key_iv: "00693b44b7461f831690455b4da4467b"
server_key_version: "encryptionkey@0"
I20260812 07:03:20.179834 26063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "162f4d9eeba348f9a9e103b57d8a85ae"
format_stamp: "Formatted at 2026-08-12 07:03:20 on dist-test-slave-lgr5"
server_key: "1153dc71f79936ca3d3ee8178dd5316f"
server_key_iv: "00693b44b7461f831690455b4da4467b"
server_key_version: "encryptionkey@0"
I20260812 07:03:20.184630 26063 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260812 07:03:20.187826 26087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:20.189047 26063 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 07:03:20.189198 26063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "162f4d9eeba348f9a9e103b57d8a85ae"
format_stamp: "Formatted at 2026-08-12 07:03:20 on dist-test-slave-lgr5"
server_key: "1153dc71f79936ca3d3ee8178dd5316f"
server_key_iv: "00693b44b7461f831690455b4da4467b"
server_key_version: "encryptionkey@0"
I20260812 07:03:20.189396 26063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:20.209036 26063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:20.212942 26063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:20.213183 26063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:20.215723 26063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:20.215828 26063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:20.216012 26063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:20.216106 26063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:03:20 dist-test-slave-lgr5 krb5kdc[25605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518200, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM
I20260812 07:03:20.237782 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:20.216049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:52629 (local address 127.11.192.254:43629)
0812 07:03:20.216259 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:20.216264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:20.217662 (+ 1398us) server_negotiation.cc:407] Connection header received
0812 07:03:20.219797 (+ 2135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:20.219803 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:20.219895 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:20.220015 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:20.222306 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.223223 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.226040 (+ 2817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.226559 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.226686 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:20.231309 (+ 4623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:20.231339 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:20.231344 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:20.231387 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:20.234537 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:20.235543 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:20.235549 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:20.235553 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:20.235680 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:20.236184 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:20.236187 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:20.236189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:20.236458 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:20.236574 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:20.237351 (+ 777us) server_negotiation.cc:299] Negotiation successful
0812 07:03:20.237508 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 07:03:20.239239 26100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:20.216749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43629 (local address 127.11.192.194:52629)
0812 07:03:20.217514 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:20.217548 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:20.219415 (+ 1867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:20.220324 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:20.220337 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:20.221164 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:20.222103 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:20.222131 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.223404 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.223411 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:20.225878 (+ 2467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:20.225896 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.226861 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.226870 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:20.227163 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:20.227853 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:20.227874 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:20.231053 (+ 3179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:20.234704 (+ 3651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:20.234713 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:20.234728 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:20.235401 (+ 673us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:20.235780 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:20.235786 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:20.235790 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:20.236061 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:20.236569 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:20.236577 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:20.237002 (+ 425us) client_negotiation.cc:769] Sending connection context
0812 07:03:20.237298 (+ 296us) client_negotiation.cc:240] Negotiation successful
0812 07:03:20.237817 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":322,"threads_started":1}
I20260812 07:03:20.257498 26220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:20.245024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43629 (local address 127.11.192.194:53923)
0812 07:03:20.245728 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:20.245752 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:20.245897 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:20.246271 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:20.246279 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:20.246695 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:20.246966 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:20.246983 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.248244 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.248250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:20.249585 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:20.249598 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.250332 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.250339 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:20.250463 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:20.251015 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:20.251030 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:20.251694 (+ 664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:20.254898 (+ 3204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:20.254904 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:20.254908 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:20.255287 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:20.255630 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:20.255634 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:20.255638 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:20.255754 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:20.256342 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:20.256347 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:20.256541 (+ 194us) client_negotiation.cc:769] Sending connection context
0812 07:03:20.256685 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 07:03:20.257062 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":438,"threads_started":1}
I20260812 07:03:20.257535 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:20.244946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:53923 (local address 127.11.192.254:43629)
0812 07:03:20.245125 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:20.245129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:20.245847 (+ 718us) server_negotiation.cc:407] Connection header received
0812 07:03:20.246079 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:20.246084 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:20.246145 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:20.246220 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:20.247112 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.248038 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.249743 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.250146 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.250251 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:20.251863 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:20.251868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:20.251871 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:20.251899 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:20.254775 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:20.255395 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:20.255400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:20.255404 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:20.255525 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:20.255851 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:20.255857 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:20.255860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:20.256221 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:20.256348 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:20.256936 (+ 588us) server_negotiation.cc:299] Negotiation successful
0812 07:03:20.257180 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 07:03:20.266122 26063 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:36263
I20260812 07:03:20.266144 26281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:36263 every 8 connection(s)
I20260812 07:03:20.267191 26063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:03:20.268939 26282 heartbeater.cc:347] Connected to a master server at 127.11.192.254:43629
I20260812 07:03:20.269136 26282 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:20.269663 26282 heartbeater.cc:511] Master 127.11.192.254:43629 requested a full tablet report, sending...
I20260812 07:03:20.270345 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 26063
I20260812 07:03:20.270533 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:03:20.271039 12035 external_mini_cluster.cc:1428] Setting key 3b79f65bddb31ce01714c23da7ff1b45
I20260812 07:03:20.271057 25695 ts_manager.cc:194] Registered new tserver with Master: 162f4d9eeba348f9a9e103b57d8a85ae (127.11.192.194:36263)
I20260812 07:03:20.271847 25695 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:52629
WARNING: no policy specified for oryx/127.11.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.195.keytab.
Entry for principal oryx/127.11.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.195.keytab.
I20260812 07:03:20.319737 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43629
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.195.keytab
--principal=oryx/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:40167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:20.632804 26289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:20.633050 26289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:20.633147 26289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:20.639946 26289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:20.640044 26289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:20.640166 26289 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:03:20.640250 26289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:03:20.649650 26289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:40167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.195.keytab
--principal=oryx/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.192.254:43629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:20.650771 26289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:20.652016 26289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:20.664363 26294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:20.664294 26295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:20.664335 26298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:20.665009 26289 server_base.cc:1034] running on GCE node
I20260812 07:03:20.665578 26289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:20.666429 26289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:20.667654 26289 hybrid_clock.cc:648] HybridClock initialized: now 1786518200667587 us; error 46 us; skew 500 ppm
I20260812 07:03:20.667936 26289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:20 dist-test-slave-lgr5 krb5kdc[25605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518200, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:20.670996 26289 init.cc:377] Logged in from keytab as oryx/127.11.192.195@KRBTEST.COM (short username oryx)
I20260812 07:03:20.672384 26289 webserver.cc:460] Webserver started at http://127.11.192.195:42261/ using document root <none> and password file <none>
I20260812 07:03:20.673031 26289 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:20.673126 26289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:20.673430 26289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:20.676348 26289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "beb93a68b67948b285012d477c962c4c"
format_stamp: "Formatted at 2026-08-12 07:03:20 on dist-test-slave-lgr5"
server_key: "3451659a97b449b0ba2346390ee243b1"
server_key_iv: "da0ce895b687f759391939217dd4ea3a"
server_key_version: "encryptionkey@0"
I20260812 07:03:20.677155 26289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "beb93a68b67948b285012d477c962c4c"
format_stamp: "Formatted at 2026-08-12 07:03:20 on dist-test-slave-lgr5"
server_key: "3451659a97b449b0ba2346390ee243b1"
server_key_iv: "da0ce895b687f759391939217dd4ea3a"
server_key_version: "encryptionkey@0"
I20260812 07:03:20.681895 26289 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:03:20.685303 26312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:20.686573 26289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:03:20.686709 26289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "beb93a68b67948b285012d477c962c4c"
format_stamp: "Formatted at 2026-08-12 07:03:20 on dist-test-slave-lgr5"
server_key: "3451659a97b449b0ba2346390ee243b1"
server_key_iv: "da0ce895b687f759391939217dd4ea3a"
server_key_version: "encryptionkey@0"
I20260812 07:03:20.686908 26289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:20.702764 26289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:20.706857 26289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:20.707114 26289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:20.709604 26289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:20.709699 26289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:20.709815 26289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:20.709873 26289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:03:20 dist-test-slave-lgr5 krb5kdc[25605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518200, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM
I20260812 07:03:20.731344 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:20.709916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:50691 (local address 127.11.192.254:43629)
0812 07:03:20.710117 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:20.710125 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:20.711461 (+ 1336us) server_negotiation.cc:407] Connection header received
0812 07:03:20.713484 (+ 2023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:20.713500 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:20.713592 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:20.713702 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:20.715658 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.716650 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.719368 (+ 2718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.719871 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.720013 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:20.724310 (+ 4297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:20.724315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:20.724318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:20.724349 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:20.727462 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:20.728321 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:20.728327 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:20.728331 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:20.728483 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:20.729106 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:20.729112 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:20.729114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:20.729501 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:20.729652 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:20.730678 (+ 1026us) server_negotiation.cc:299] Negotiation successful
0812 07:03:20.730948 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:03:20.732817 26329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:20.710442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43629 (local address 127.11.192.195:50691)
0812 07:03:20.711279 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:20.711330 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:20.713219 (+ 1889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:20.713946 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:20.713957 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:20.714656 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:20.715476 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:20.715501 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.716810 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.716819 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:20.719182 (+ 2363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:20.719198 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.720141 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.720148 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:20.720443 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:20.721196 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:20.721235 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:20.724118 (+ 2883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:20.727598 (+ 3480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:20.727609 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:20.727623 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:20.728198 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:20.728615 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:20.728620 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:20.728623 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:20.728928 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:20.729730 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:20.729739 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:20.730194 (+ 455us) client_negotiation.cc:769] Sending connection context
0812 07:03:20.730546 (+ 352us) client_negotiation.cc:240] Negotiation successful
0812 07:03:20.731081 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":336,"threads_started":1}
I20260812 07:03:20.752102 26289 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:34345
I20260812 07:03:20.752172 26526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:34345 every 8 connection(s)
I20260812 07:03:20.753244 26289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:03:20.757987 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 26289
I20260812 07:03:20.757998 26527 heartbeater.cc:347] Connected to a master server at 127.11.192.254:43629
I20260812 07:03:20.758176 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:03:20.758220 26527 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:20.758704 12035 external_mini_cluster.cc:1428] Setting key 1e7b4fb0bd9e639a90096c1324c8699b
I20260812 07:03:20.758999 26527 heartbeater.cc:511] Master 127.11.192.254:43629 requested a full tablet report, sending...
I20260812 07:03:20.762669 25694 ts_manager.cc:194] Registered new tserver with Master: beb93a68b67948b285012d477c962c4c (127.11.192.195:34345)
I20260812 07:03:20.763263 26509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:20.746544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43629 (local address 127.11.192.195:57529)
0812 07:03:20.748944 (+ 2400us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:20.748964 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:20.749089 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:20.749440 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:20.749445 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:20.750002 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:20.750397 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:20.750410 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.751595 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.751600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:20.752896 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:20.752909 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.753641 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.753648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:20.753822 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:20.754413 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:20.754429 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:20.755129 (+ 700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:20.760337 (+ 5208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:20.760342 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:20.760345 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:20.760859 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:20.761337 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:20.761343 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:20.761347 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:20.761522 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:20.762171 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:20.762177 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:20.762377 (+ 200us) client_negotiation.cc:769] Sending connection context
0812 07:03:20.762519 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 07:03:20.762829 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2261,"thread_start_us":2123,"threads_started":1}
I20260812 07:03:20.763270 26057 heartbeater.cc:503] Master 127.11.192.254:43629 was elected leader, sending a full tablet report...
I20260812 07:03:20.763707 25694 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:50691
I20260812 07:03:20.764031 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:20.746408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:57529 (local address 127.11.192.254:43629)
0812 07:03:20.746597 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:20.746602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:20.749089 (+ 2487us) server_negotiation.cc:407] Connection header received
0812 07:03:20.749209 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:20.749232 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:20.749303 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:20.749386 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:20.750537 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.751416 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.753031 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.753457 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.753570 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:20.755307 (+ 1737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:20.755312 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:20.755315 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:20.755343 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:20.760193 (+ 4850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:20.761024 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:20.761028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:20.761031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:20.761186 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:20.761667 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:20.761673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:20.761677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:20.762025 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:20.763506 (+ 1481us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:20.763614 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 07:03:20.763788 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":64}
I20260812 07:03:20.773720 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 07:03:20 dist-test-slave-lgr5 krb5kdc[25605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518200, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 07:03:20 dist-test-slave-lgr5 krb5kdc[25605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:20 dist-test-slave-lgr5 krb5kdc[25605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 07:03:20.827132 25861 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:20.817446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53212 (local address 127.11.192.254:43629)
0812 07:03:20.817658 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:20.817664 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:20.817799 (+ 135us) server_negotiation.cc:407] Connection header received
0812 07:03:20.817984 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:20.817996 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:20.818073 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:20.818161 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:20.819462 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.820234 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.821452 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.821857 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.821967 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:20.826888 (+ 4921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53212: BlockingRecv error: recv got EOF from 127.0.0.1:53212 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 12 07:03:20 dist-test-slave-lgr5 krb5kdc[25605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518200, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM
I20260812 07:03:20.845603 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:20.830244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53220 (local address 127.11.192.254:43629)
0812 07:03:20.830392 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:20.830396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:20.830704 (+ 308us) server_negotiation.cc:407] Connection header received
0812 07:03:20.830821 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:20.830824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:20.830882 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:20.830958 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:20.832020 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.832768 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.833928 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.834390 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.834508 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:20.841367 (+ 6859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:20.841372 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:20.841375 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:20.841405 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:20.843663 (+ 2258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:20.844232 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:20.844236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:20.844238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:20.844348 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:20.844614 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:20.844617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:20.844619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:20.844857 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:20.845071 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:20.845225 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 07:03:20.845382 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:03:20.849015 25694 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:03:20.855764 25694 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:03:20.887946 26559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:20.867026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45256 (local address 127.11.192.194:36263)
0812 07:03:20.880033 (+ 13007us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:20.880042 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:20.880072 (+ 30us) server_negotiation.cc:407] Connection header received
0812 07:03:20.880204 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:20.880211 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:20.880456 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:20.880683 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:20.881627 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.883154 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.885835 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.886926 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.887081 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:20.887258 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:20.887512 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 07:03:20.887670 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12868,"thread_start_us":12763,"threads_started":1}
I20260812 07:03:20.887984 26561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:20.866836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:36263 (local address 127.0.0.1:45256)
0812 07:03:20.877099 (+ 10263us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:20.877142 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:20.877322 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:20.880893 (+ 3571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:20.880902 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:20.880946 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:20.881399 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:20.881410 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.883288 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.883294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:20.885633 (+ 2339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:20.885647 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.887108 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.887114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:20.887291 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:20.887333 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 07:03:20.887437 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 07:03:20.887609 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9983,"mutex_wait_us":208,"thread_start_us":11919,"threads_started":1}
I20260812 07:03:20.889065 26558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:20.867057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:34345 (local address 127.0.0.1:54354)
0812 07:03:20.878547 (+ 11490us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:20.878565 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:20.878685 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:20.881491 (+ 2806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:20.881495 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:20.881521 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:20.881887 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:20.881900 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.883689 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.883694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:20.885826 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:20.885842 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.888291 (+ 2449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.888298 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:20.888451 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:20.888487 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 07:03:20.888555 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 07:03:20.888674 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11352,"mutex_wait_us":71,"thread_start_us":13986,"threads_started":1}
I20260812 07:03:20.889323 26563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:20.868419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56944 (local address 127.11.192.193:40691)
0812 07:03:20.880394 (+ 11975us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:20.880401 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:20.881790 (+ 1389us) server_negotiation.cc:407] Connection header received
0812 07:03:20.881894 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:20.881901 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:20.882112 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:20.882291 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:20.883015 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.884424 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.886630 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.888042 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.888226 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:20.888389 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:20.888887 (+ 498us) server_negotiation.cc:299] Negotiation successful
0812 07:03:20.889052 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11830,"thread_start_us":11690,"threads_started":1}
I20260812 07:03:20.890239 26562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:20.867873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54354 (local address 127.11.192.195:34345)
0812 07:03:20.880934 (+ 13061us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:20.880942 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:20.880971 (+ 29us) server_negotiation.cc:407] Connection header received
0812 07:03:20.881079 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:20.881085 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:20.881295 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:20.881488 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:20.882121 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.883564 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.886648 (+ 3084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.888043 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.889546 (+ 1503us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:20.889701 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:20.889771 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 07:03:20.889915 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12894,"thread_start_us":12698,"threads_started":1}
I20260812 07:03:20.892525 26560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:20.867443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:40691 (local address 127.0.0.1:56944)
0812 07:03:20.881355 (+ 13912us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:20.881379 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:20.881498 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:20.882526 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:20.882530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:20.882554 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:20.882847 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:20.882855 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.884572 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.884578 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:20.886476 (+ 1898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:20.886488 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:20.888537 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:20.888544 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:20.888701 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:20.888730 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 07:03:20.888822 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 07:03:20.888935 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13791,"mutex_wait_us":42,"thread_start_us":9239,"threads_started":1}
I20260812 07:03:20.891989 26166 tablet_service.cc:1467] Processing CreateTablet for tablet 212f046014b54a278b3ce79147eb5d4a (DEFAULT_TABLE table=test-table [id=c2b672ad4518405fbea2fd5fcb027056]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:03:20.893460 26166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 212f046014b54a278b3ce79147eb5d4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:20.893493 26411 tablet_service.cc:1467] Processing CreateTablet for tablet 212f046014b54a278b3ce79147eb5d4a (DEFAULT_TABLE table=test-table [id=c2b672ad4518405fbea2fd5fcb027056]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:03:20.894660 26411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 212f046014b54a278b3ce79147eb5d4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:20.896193 25932 tablet_service.cc:1467] Processing CreateTablet for tablet 212f046014b54a278b3ce79147eb5d4a (DEFAULT_TABLE table=test-table [id=c2b672ad4518405fbea2fd5fcb027056]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:03:20.897534 25932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 212f046014b54a278b3ce79147eb5d4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:20.905678 26564 tablet_bootstrap.cc:492] T 212f046014b54a278b3ce79147eb5d4a P beb93a68b67948b285012d477c962c4c: Bootstrap starting.
I20260812 07:03:20.908757 26565 tablet_bootstrap.cc:492] T 212f046014b54a278b3ce79147eb5d4a P 162f4d9eeba348f9a9e103b57d8a85ae: Bootstrap starting.
I20260812 07:03:20.909574 26564 tablet_bootstrap.cc:654] T 212f046014b54a278b3ce79147eb5d4a P beb93a68b67948b285012d477c962c4c: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:20.910816 26564 log.cc:824] T 212f046014b54a278b3ce79147eb5d4a P beb93a68b67948b285012d477c962c4c: Log is configured to *not* fsync() on all Append() calls
I20260812 07:03:20.911059 26566 tablet_bootstrap.cc:492] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9: Bootstrap starting.
I20260812 07:03:20.911983 26565 tablet_bootstrap.cc:654] T 212f046014b54a278b3ce79147eb5d4a P 162f4d9eeba348f9a9e103b57d8a85ae: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:20.913025 26565 log.cc:824] T 212f046014b54a278b3ce79147eb5d4a P 162f4d9eeba348f9a9e103b57d8a85ae: Log is configured to *not* fsync() on all Append() calls
I20260812 07:03:20.914178 26566 tablet_bootstrap.cc:654] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:20.914315 26564 tablet_bootstrap.cc:492] T 212f046014b54a278b3ce79147eb5d4a P beb93a68b67948b285012d477c962c4c: No bootstrap required, opened a new log
I20260812 07:03:20.914661 26564 ts_tablet_manager.cc:1397] T 212f046014b54a278b3ce79147eb5d4a P beb93a68b67948b285012d477c962c4c: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260812 07:03:20.915215 26566 log.cc:824] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9: Log is configured to *not* fsync() on all Append() calls
I20260812 07:03:20.917685 26565 tablet_bootstrap.cc:492] T 212f046014b54a278b3ce79147eb5d4a P 162f4d9eeba348f9a9e103b57d8a85ae: No bootstrap required, opened a new log
I20260812 07:03:20.917915 26565 ts_tablet_manager.cc:1397] T 212f046014b54a278b3ce79147eb5d4a P 162f4d9eeba348f9a9e103b57d8a85ae: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260812 07:03:20.917929 26566 tablet_bootstrap.cc:492] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9: No bootstrap required, opened a new log
I20260812 07:03:20.918169 26566 ts_tablet_manager.cc:1397] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260812 07:03:20.924530 26564 raft_consensus.cc:348] T 212f046014b54a278b3ce79147eb5d4a P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } } peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } }
I20260812 07:03:20.924934 26564 raft_consensus.cc:374] T 212f046014b54a278b3ce79147eb5d4a P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:20.925038 26564 raft_consensus.cc:729] T 212f046014b54a278b3ce79147eb5d4a P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beb93a68b67948b285012d477c962c4c, State: Initialized, Role: FOLLOWER
I20260812 07:03:20.924815 26565 raft_consensus.cc:348] T 212f046014b54a278b3ce79147eb5d4a P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } } peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } }
I20260812 07:03:20.925179 26565 raft_consensus.cc:374] T 212f046014b54a278b3ce79147eb5d4a P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:20.925285 26565 raft_consensus.cc:729] T 212f046014b54a278b3ce79147eb5d4a P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 162f4d9eeba348f9a9e103b57d8a85ae, State: Initialized, Role: FOLLOWER
I20260812 07:03:20.925701 26564 consensus_queue.cc:260] T 212f046014b54a278b3ce79147eb5d4a P beb93a68b67948b285012d477c962c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } } peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } }
I20260812 07:03:20.925788 26565 consensus_queue.cc:260] T 212f046014b54a278b3ce79147eb5d4a P 162f4d9eeba348f9a9e103b57d8a85ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } } peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } }
I20260812 07:03:20.928009 26566 raft_consensus.cc:348] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } } peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } }
I20260812 07:03:20.928503 26566 raft_consensus.cc:374] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:20.928650 26566 raft_consensus.cc:729] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b70782d5814ce8a9beb0f8712762c9, State: Initialized, Role: FOLLOWER
I20260812 07:03:20.928690 26282 heartbeater.cc:503] Master 127.11.192.254:43629 was elected leader, sending a full tablet report...
I20260812 07:03:20.928881 26564 ts_tablet_manager.cc:1428] T 212f046014b54a278b3ce79147eb5d4a P beb93a68b67948b285012d477c962c4c: Time spent starting tablet: real 0.014s user 0.008s sys 0.005s
I20260812 07:03:20.929160 26565 ts_tablet_manager.cc:1428] T 212f046014b54a278b3ce79147eb5d4a P 162f4d9eeba348f9a9e103b57d8a85ae: Time spent starting tablet: real 0.011s user 0.000s sys 0.012s
I20260812 07:03:20.929425 26566 consensus_queue.cc:260] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } } peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } }
I20260812 07:03:20.929684 26527 heartbeater.cc:503] Master 127.11.192.254:43629 was elected leader, sending a full tablet report...
I20260812 07:03:20.931922 26566 ts_tablet_manager.cc:1428] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
W20260812 07:03:21.007542 26059 tablet.cc:2369] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:03:21.013223 26528 tablet.cc:2369] T 212f046014b54a278b3ce79147eb5d4a P beb93a68b67948b285012d477c962c4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:03:21.018931 26283 tablet.cc:2369] T 212f046014b54a278b3ce79147eb5d4a P 162f4d9eeba348f9a9e103b57d8a85ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:03:21.147079 26582 raft_consensus.cc:482] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:03:21.147368 26582 raft_consensus.cc:504] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } } peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } }
I20260812 07:03:21.149077 26582 leader_election.cc:288] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 162f4d9eeba348f9a9e103b57d8a85ae (127.11.192.194:36263), beb93a68b67948b285012d477c962c4c (127.11.192.195:34345)
I20260812 07:03:21.155453 26559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:21.149582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:55977 (local address 127.11.192.194:36263)
0812 07:03:21.149792 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:21.149798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:21.150139 (+ 341us) server_negotiation.cc:407] Connection header received
0812 07:03:21.150337 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:21.150341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:21.150402 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:21.150477 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:21.151065 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.151803 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.153700 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.154567 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.154713 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:21.154788 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:21.155078 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 07:03:21.155203 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 07:03:21.155531 26562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:21.149573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:46959 (local address 127.11.192.195:34345)
0812 07:03:21.149778 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:21.149785 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:21.150616 (+ 831us) server_negotiation.cc:407] Connection header received
0812 07:03:21.150780 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:21.150785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:21.150859 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:21.150930 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:21.151435 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.152161 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.153857 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.154676 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.154832 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:21.154901 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:21.155181 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 07:03:21.155298 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 07:03:21.155627 26583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:21.149307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:36263 (local address 127.11.192.193:55977)
0812 07:03:21.150021 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:21.150037 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:21.150192 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:21.150520 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:21.150525 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:21.150553 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:21.150900 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:21.150914 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.151932 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.151936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:21.153541 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:21.153554 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.154757 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.154764 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:21.154915 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:21.154941 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 07:03:21.155024 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 07:03:21.155163 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"mutex_wait_us":67,"spinlock_wait_cycles":18304,"thread_start_us":335,"threads_started":1}
I20260812 07:03:21.155721 26584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:21.149307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:34345 (local address 127.11.192.193:46959)
0812 07:03:21.150493 (+ 1186us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:21.150509 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:21.150648 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:21.150969 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:21.150973 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:21.150997 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:21.151287 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:21.151296 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.152294 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.152298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:21.153706 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:21.153719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.154895 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.154902 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:21.155025 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:21.155065 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 07:03:21.155141 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 07:03:21.155259 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1052,"mutex_wait_us":487,"thread_start_us":288,"threads_started":1}
I20260812 07:03:21.156890 26216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "212f046014b54a278b3ce79147eb5d4a" candidate_uuid: "e2b70782d5814ce8a9beb0f8712762c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" is_pre_election: true
I20260812 07:03:21.156904 26440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "212f046014b54a278b3ce79147eb5d4a" candidate_uuid: "e2b70782d5814ce8a9beb0f8712762c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "beb93a68b67948b285012d477c962c4c" is_pre_election: true
I20260812 07:03:21.157485 26216 raft_consensus.cc:2454] T 212f046014b54a278b3ce79147eb5d4a P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b70782d5814ce8a9beb0f8712762c9 in term 0.
I20260812 07:03:21.157485 26440 raft_consensus.cc:2454] T 212f046014b54a278b3ce79147eb5d4a P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b70782d5814ce8a9beb0f8712762c9 in term 0.
I20260812 07:03:21.158175 25854 leader_election.cc:302] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 162f4d9eeba348f9a9e103b57d8a85ae, e2b70782d5814ce8a9beb0f8712762c9; no voters:
I20260812 07:03:21.158563 26582 raft_consensus.cc:2781] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:03:21.158694 26582 raft_consensus.cc:482] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:03:21.158788 26582 raft_consensus.cc:3037] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:21.160982 26582 raft_consensus.cc:504] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } } peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } }
I20260812 07:03:21.161525 26582 leader_election.cc:288] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9 [CANDIDATE]: Term 1 election: Requested vote from peers 162f4d9eeba348f9a9e103b57d8a85ae (127.11.192.194:36263), beb93a68b67948b285012d477c962c4c (127.11.192.195:34345)
I20260812 07:03:21.162001 26216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "212f046014b54a278b3ce79147eb5d4a" candidate_uuid: "e2b70782d5814ce8a9beb0f8712762c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "162f4d9eeba348f9a9e103b57d8a85ae"
I20260812 07:03:21.161991 26440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "212f046014b54a278b3ce79147eb5d4a" candidate_uuid: "e2b70782d5814ce8a9beb0f8712762c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "beb93a68b67948b285012d477c962c4c"
I20260812 07:03:21.162195 26216 raft_consensus.cc:3037] T 212f046014b54a278b3ce79147eb5d4a P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:21.162195 26440 raft_consensus.cc:3037] T 212f046014b54a278b3ce79147eb5d4a P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:21.165328 26440 raft_consensus.cc:2454] T 212f046014b54a278b3ce79147eb5d4a P beb93a68b67948b285012d477c962c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b70782d5814ce8a9beb0f8712762c9 in term 1.
I20260812 07:03:21.165328 26216 raft_consensus.cc:2454] T 212f046014b54a278b3ce79147eb5d4a P 162f4d9eeba348f9a9e103b57d8a85ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b70782d5814ce8a9beb0f8712762c9 in term 1.
I20260812 07:03:21.165735 25854 leader_election.cc:302] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 162f4d9eeba348f9a9e103b57d8a85ae, beb93a68b67948b285012d477c962c4c, e2b70782d5814ce8a9beb0f8712762c9; no voters:
I20260812 07:03:21.166035 26582 raft_consensus.cc:2781] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:03:21.166431 26582 raft_consensus.cc:686] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9 [term 1 LEADER]: Becoming Leader. State: Replica: e2b70782d5814ce8a9beb0f8712762c9, State: Running, Role: LEADER
I20260812 07:03:21.166898 26582 consensus_queue.cc:237] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } } peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } }
I20260812 07:03:21.171200 25695 catalog_manager.cc:5495] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2b70782d5814ce8a9beb0f8712762c9 (127.11.192.193). New cstate: current_term: 1 leader_uuid: "e2b70782d5814ce8a9beb0f8712762c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } health_report { overall_health: HEALTHY } } }
I20260812 07:03:21.254346 26563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:21.249691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56946 (local address 127.11.192.193:40691)
0812 07:03:21.249918 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:21.249923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:21.249941 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:03:21.250130 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:21.250134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:21.250212 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:21.250332 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:03:21.250603 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.251289 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.252533 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.252926 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.253013 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:21.253335 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:03:21.253860 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:03:21.253959 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:21.254029 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 07:03:21.254091 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:03:21.262195 26440 raft_consensus.cc:1261] T 212f046014b54a278b3ce79147eb5d4a P beb93a68b67948b285012d477c962c4c [term 1 FOLLOWER]: Refusing update from remote peer e2b70782d5814ce8a9beb0f8712762c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:03:21.262194 26216 raft_consensus.cc:1261] T 212f046014b54a278b3ce79147eb5d4a P 162f4d9eeba348f9a9e103b57d8a85ae [term 1 FOLLOWER]: Refusing update from remote peer e2b70782d5814ce8a9beb0f8712762c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:03:21.263218 26585 consensus_queue.cc:1035] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:03:21.263559 26582 consensus_queue.cc:1035] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:03:21.276245 26590 mvcc.cc:205] Tried to move back new op lower bound from 7317578552360210432 to 7317578551983063040. Current Snapshot: MvccSnapshot[applied={T|T < 7317578552360210432}]
I20260812 07:03:21.296834 25693 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:03:21.297147 25693 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:03:21.301054 25693 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883: Sending DeleteTablet for 3 replicas of tablet 212f046014b54a278b3ce79147eb5d4a
I20260812 07:03:21.302062 26166 tablet_service.cc:1514] Processing DeleteTablet for tablet 212f046014b54a278b3ce79147eb5d4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:03:21 UTC) from {username='oryx', principal='oryx/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:45256
I20260812 07:03:21.302172 26411 tablet_service.cc:1514] Processing DeleteTablet for tablet 212f046014b54a278b3ce79147eb5d4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:03:21 UTC) from {username='oryx', principal='oryx/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:54354
I20260812 07:03:21.302544 25932 tablet_service.cc:1514] Processing DeleteTablet for tablet 212f046014b54a278b3ce79147eb5d4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:03:21 UTC) from {username='oryx', principal='oryx/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:56944
I20260812 07:03:21.305903 25693 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:03:21.306674 25693 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:03:21.317601 25930 tablet_service.cc:1467] Processing CreateTablet for tablet 3bd2f0fc398f44248483fc239933b201 (DEFAULT_TABLE table=test-table [id=79b465a35fc8495d84312b33f436c29c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:03:21.317950 26165 tablet_service.cc:1467] Processing CreateTablet for tablet 3bd2f0fc398f44248483fc239933b201 (DEFAULT_TABLE table=test-table [id=79b465a35fc8495d84312b33f436c29c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:03:21.318212 25930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bd2f0fc398f44248483fc239933b201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:21.318504 26165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bd2f0fc398f44248483fc239933b201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:21.320384 26606 tablet_replica.cc:330] T 212f046014b54a278b3ce79147eb5d4a P 162f4d9eeba348f9a9e103b57d8a85ae: stopping tablet replica
I20260812 07:03:21.320740 26410 tablet_service.cc:1467] Processing CreateTablet for tablet 3bd2f0fc398f44248483fc239933b201 (DEFAULT_TABLE table=test-table [id=79b465a35fc8495d84312b33f436c29c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:03:21.321132 26606 raft_consensus.cc:2229] T 212f046014b54a278b3ce79147eb5d4a P 162f4d9eeba348f9a9e103b57d8a85ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:03:21.321230 26410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bd2f0fc398f44248483fc239933b201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:21.321826 26606 raft_consensus.cc:2258] T 212f046014b54a278b3ce79147eb5d4a P 162f4d9eeba348f9a9e103b57d8a85ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:03:21.322855 26605 tablet_replica.cc:330] T 212f046014b54a278b3ce79147eb5d4a P beb93a68b67948b285012d477c962c4c: stopping tablet replica
I20260812 07:03:21.323052 26608 tablet_replica.cc:330] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9: stopping tablet replica
I20260812 07:03:21.323473 26608 raft_consensus.cc:2229] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:03:21.323482 26605 raft_consensus.cc:2229] T 212f046014b54a278b3ce79147eb5d4a P beb93a68b67948b285012d477c962c4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:03:21.324095 26605 raft_consensus.cc:2258] T 212f046014b54a278b3ce79147eb5d4a P beb93a68b67948b285012d477c962c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:03:21.325049 26565 tablet_bootstrap.cc:492] T 3bd2f0fc398f44248483fc239933b201 P 162f4d9eeba348f9a9e103b57d8a85ae: Bootstrap starting.
I20260812 07:03:21.325636 26608 raft_consensus.cc:2258] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:03:21.326337 26605 ts_tablet_manager.cc:1905] T 212f046014b54a278b3ce79147eb5d4a P beb93a68b67948b285012d477c962c4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:03:21.326625 26565 tablet_bootstrap.cc:654] T 3bd2f0fc398f44248483fc239933b201 P 162f4d9eeba348f9a9e103b57d8a85ae: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:21.329602 26608 ts_tablet_manager.cc:1905] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:03:21.333189 26606 ts_tablet_manager.cc:1905] T 212f046014b54a278b3ce79147eb5d4a P 162f4d9eeba348f9a9e103b57d8a85ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:03:21.333505 26605 ts_tablet_manager.cc:1918] T 212f046014b54a278b3ce79147eb5d4a P beb93a68b67948b285012d477c962c4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:03:21.333686 26605 log.cc:1196] T 212f046014b54a278b3ce79147eb5d4a P beb93a68b67948b285012d477c962c4c: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/212f046014b54a278b3ce79147eb5d4a
I20260812 07:03:21.334215 26605 ts_tablet_manager.cc:1939] T 212f046014b54a278b3ce79147eb5d4a P beb93a68b67948b285012d477c962c4c: Deleting consensus metadata
I20260812 07:03:21.336180 25665 catalog_manager.cc:4841] TS beb93a68b67948b285012d477c962c4c (127.11.192.195:34345): tablet 212f046014b54a278b3ce79147eb5d4a (table test-table [id=c2b672ad4518405fbea2fd5fcb027056]) successfully deleted
I20260812 07:03:21.337507 26564 tablet_bootstrap.cc:492] T 3bd2f0fc398f44248483fc239933b201 P beb93a68b67948b285012d477c962c4c: Bootstrap starting.
I20260812 07:03:21.339172 26564 tablet_bootstrap.cc:654] T 3bd2f0fc398f44248483fc239933b201 P beb93a68b67948b285012d477c962c4c: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:21.340767 26566 tablet_bootstrap.cc:492] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9: Bootstrap starting.
I20260812 07:03:21.342511 26566 tablet_bootstrap.cc:654] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:21.347100 26606 ts_tablet_manager.cc:1918] T 212f046014b54a278b3ce79147eb5d4a P 162f4d9eeba348f9a9e103b57d8a85ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:03:21.347261 26606 log.cc:1196] T 212f046014b54a278b3ce79147eb5d4a P 162f4d9eeba348f9a9e103b57d8a85ae: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/212f046014b54a278b3ce79147eb5d4a
I20260812 07:03:21.347767 26606 ts_tablet_manager.cc:1939] T 212f046014b54a278b3ce79147eb5d4a P 162f4d9eeba348f9a9e103b57d8a85ae: Deleting consensus metadata
I20260812 07:03:21.347800 26608 ts_tablet_manager.cc:1918] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:03:21.348230 26608 log.cc:1196] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-0/wal/wals/212f046014b54a278b3ce79147eb5d4a
I20260812 07:03:21.348974 26608 ts_tablet_manager.cc:1939] T 212f046014b54a278b3ce79147eb5d4a P e2b70782d5814ce8a9beb0f8712762c9: Deleting consensus metadata
I20260812 07:03:21.351300 25663 catalog_manager.cc:4841] TS e2b70782d5814ce8a9beb0f8712762c9 (127.11.192.193:40691): tablet 212f046014b54a278b3ce79147eb5d4a (table test-table [id=c2b672ad4518405fbea2fd5fcb027056]) successfully deleted
I20260812 07:03:21.352980 26565 tablet_bootstrap.cc:492] T 3bd2f0fc398f44248483fc239933b201 P 162f4d9eeba348f9a9e103b57d8a85ae: No bootstrap required, opened a new log
I20260812 07:03:21.353127 26565 ts_tablet_manager.cc:1397] T 3bd2f0fc398f44248483fc239933b201 P 162f4d9eeba348f9a9e103b57d8a85ae: Time spent bootstrapping tablet: real 0.028s user 0.006s sys 0.003s
I20260812 07:03:21.353538 26564 tablet_bootstrap.cc:492] T 3bd2f0fc398f44248483fc239933b201 P beb93a68b67948b285012d477c962c4c: No bootstrap required, opened a new log
I20260812 07:03:21.353688 26564 ts_tablet_manager.cc:1397] T 3bd2f0fc398f44248483fc239933b201 P beb93a68b67948b285012d477c962c4c: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.007s
I20260812 07:03:21.353921 26565 raft_consensus.cc:348] T 3bd2f0fc398f44248483fc239933b201 P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:21.354177 26565 raft_consensus.cc:374] T 3bd2f0fc398f44248483fc239933b201 P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:21.354283 26565 raft_consensus.cc:729] T 3bd2f0fc398f44248483fc239933b201 P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 162f4d9eeba348f9a9e103b57d8a85ae, State: Initialized, Role: FOLLOWER
I20260812 07:03:21.354429 26564 raft_consensus.cc:348] T 3bd2f0fc398f44248483fc239933b201 P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:21.354585 26565 consensus_queue.cc:260] T 3bd2f0fc398f44248483fc239933b201 P 162f4d9eeba348f9a9e103b57d8a85ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:21.354652 26564 raft_consensus.cc:374] T 3bd2f0fc398f44248483fc239933b201 P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:21.354815 26564 raft_consensus.cc:729] T 3bd2f0fc398f44248483fc239933b201 P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beb93a68b67948b285012d477c962c4c, State: Initialized, Role: FOLLOWER
I20260812 07:03:21.355144 26564 consensus_queue.cc:260] T 3bd2f0fc398f44248483fc239933b201 P beb93a68b67948b285012d477c962c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:21.355753 26564 ts_tablet_manager.cc:1428] T 3bd2f0fc398f44248483fc239933b201 P beb93a68b67948b285012d477c962c4c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260812 07:03:21.355386 26565 ts_tablet_manager.cc:1428] T 3bd2f0fc398f44248483fc239933b201 P 162f4d9eeba348f9a9e103b57d8a85ae: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260812 07:03:21.357426 26566 tablet_bootstrap.cc:492] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9: No bootstrap required, opened a new log
I20260812 07:03:21.357563 26566 ts_tablet_manager.cc:1397] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.009s
I20260812 07:03:21.358325 26566 raft_consensus.cc:348] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:21.358592 26566 raft_consensus.cc:374] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:21.358705 26566 raft_consensus.cc:729] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b70782d5814ce8a9beb0f8712762c9, State: Initialized, Role: FOLLOWER
I20260812 07:03:21.358999 26566 consensus_queue.cc:260] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:21.360394 26566 ts_tablet_manager.cc:1428] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 07:03:21.361550 25667 catalog_manager.cc:4841] TS 162f4d9eeba348f9a9e103b57d8a85ae (127.11.192.194:36263): tablet 212f046014b54a278b3ce79147eb5d4a (table test-table [id=c2b672ad4518405fbea2fd5fcb027056]) successfully deleted
I20260812 07:03:21.692042 26601 raft_consensus.cc:482] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:03:21.692315 26601 raft_consensus.cc:504] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:21.693133 26601 leader_election.cc:288] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 162f4d9eeba348f9a9e103b57d8a85ae (127.11.192.194:36263), beb93a68b67948b285012d477c962c4c (127.11.192.195:34345)
I20260812 07:03:21.693706 26216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bd2f0fc398f44248483fc239933b201" candidate_uuid: "e2b70782d5814ce8a9beb0f8712762c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" is_pre_election: true
I20260812 07:03:21.693717 26440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bd2f0fc398f44248483fc239933b201" candidate_uuid: "e2b70782d5814ce8a9beb0f8712762c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "beb93a68b67948b285012d477c962c4c" is_pre_election: true
I20260812 07:03:21.693938 26216 raft_consensus.cc:2454] T 3bd2f0fc398f44248483fc239933b201 P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b70782d5814ce8a9beb0f8712762c9 in term 0.
I20260812 07:03:21.693955 26440 raft_consensus.cc:2454] T 3bd2f0fc398f44248483fc239933b201 P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b70782d5814ce8a9beb0f8712762c9 in term 0.
I20260812 07:03:21.694399 25850 leader_election.cc:302] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: beb93a68b67948b285012d477c962c4c, e2b70782d5814ce8a9beb0f8712762c9; no voters:
I20260812 07:03:21.694712 26601 raft_consensus.cc:2781] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:03:21.694839 26601 raft_consensus.cc:482] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:03:21.694929 26601 raft_consensus.cc:3037] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:21.697029 26601 raft_consensus.cc:504] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:21.697521 26601 leader_election.cc:288] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9 [CANDIDATE]: Term 1 election: Requested vote from peers 162f4d9eeba348f9a9e103b57d8a85ae (127.11.192.194:36263), beb93a68b67948b285012d477c962c4c (127.11.192.195:34345)
I20260812 07:03:21.697952 26216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bd2f0fc398f44248483fc239933b201" candidate_uuid: "e2b70782d5814ce8a9beb0f8712762c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "162f4d9eeba348f9a9e103b57d8a85ae"
I20260812 07:03:21.697963 26440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bd2f0fc398f44248483fc239933b201" candidate_uuid: "e2b70782d5814ce8a9beb0f8712762c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "beb93a68b67948b285012d477c962c4c"
I20260812 07:03:21.698123 26216 raft_consensus.cc:3037] T 3bd2f0fc398f44248483fc239933b201 P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:21.698151 26440 raft_consensus.cc:3037] T 3bd2f0fc398f44248483fc239933b201 P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:21.700158 26440 raft_consensus.cc:2454] T 3bd2f0fc398f44248483fc239933b201 P beb93a68b67948b285012d477c962c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b70782d5814ce8a9beb0f8712762c9 in term 1.
I20260812 07:03:21.700251 26216 raft_consensus.cc:2454] T 3bd2f0fc398f44248483fc239933b201 P 162f4d9eeba348f9a9e103b57d8a85ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b70782d5814ce8a9beb0f8712762c9 in term 1.
I20260812 07:03:21.700501 25850 leader_election.cc:302] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: beb93a68b67948b285012d477c962c4c, e2b70782d5814ce8a9beb0f8712762c9; no voters:
I20260812 07:03:21.700742 26601 raft_consensus.cc:2781] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:03:21.700865 26601 raft_consensus.cc:686] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9 [term 1 LEADER]: Becoming Leader. State: Replica: e2b70782d5814ce8a9beb0f8712762c9, State: Running, Role: LEADER
I20260812 07:03:21.701124 26601 consensus_queue.cc:237] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:21.703508 25695 catalog_manager.cc:5495] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2b70782d5814ce8a9beb0f8712762c9 (127.11.192.193). New cstate: current_term: 1 leader_uuid: "e2b70782d5814ce8a9beb0f8712762c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } health_report { overall_health: UNKNOWN } } }
Aug 12 07:03:21 dist-test-slave-lgr5 krb5kdc[25605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518199, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.254@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM
I20260812 07:03:21.814611 26623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:21.802081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43629 (local address 127.0.0.1:53232)
0812 07:03:21.802823 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:21.802842 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:21.802955 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:21.803249 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:21.803252 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:21.803842 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:21.804056 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:21.804065 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.805098 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.805102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:21.806031 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:21.806041 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.806686 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.806691 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:21.806804 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:21.807364 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:21.807386 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:21.809544 (+ 2158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:21.812516 (+ 2972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:21.812526 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:21.812532 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:21.812913 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:21.813210 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:21.813213 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:21.813216 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:21.813359 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:21.813819 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:21.813825 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:21.813960 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 07:03:21.814110 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 07:03:21.814313 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":368,"threads_started":1}
I20260812 07:03:21.814720 26622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:21.802081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53232 (local address 127.11.192.254:43629)
0812 07:03:21.802759 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:21.802764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:21.802908 (+ 144us) server_negotiation.cc:407] Connection header received
0812 07:03:21.803067 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:21.803071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:21.803134 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:21.803211 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:21.804209 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.804977 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.806140 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.806560 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.806644 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:21.809779 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:21.809785 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:21.809789 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:21.809824 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:21.812397 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:21.813005 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:21.813009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:21.813011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:21.813124 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:21.813441 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:21.813445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:21.813446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:21.813715 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:21.813821 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:21.814207 (+ 386us) server_negotiation.cc:299] Negotiation successful
0812 07:03:21.814431 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":541,"thread_start_us":326,"threads_started":1}
I20260812 07:03:21.819044 25695 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:53232:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 07:03:21.819989 25695 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:03:21.825876 25932 tablet_service.cc:1467] Processing CreateTablet for tablet adaeaddfb1e94ea1baf46d62657ef3bf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2941b37a9c8242c686a466c501bcfb61]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:03:21.826081 26165 tablet_service.cc:1467] Processing CreateTablet for tablet adaeaddfb1e94ea1baf46d62657ef3bf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2941b37a9c8242c686a466c501bcfb61]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:03:21.826457 25932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adaeaddfb1e94ea1baf46d62657ef3bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:21.826305 26410 tablet_service.cc:1467] Processing CreateTablet for tablet adaeaddfb1e94ea1baf46d62657ef3bf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2941b37a9c8242c686a466c501bcfb61]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:03:21.826607 26165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adaeaddfb1e94ea1baf46d62657ef3bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:21.826822 26410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adaeaddfb1e94ea1baf46d62657ef3bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:21.832278 26565 tablet_bootstrap.cc:492] T adaeaddfb1e94ea1baf46d62657ef3bf P 162f4d9eeba348f9a9e103b57d8a85ae: Bootstrap starting.
I20260812 07:03:21.833307 26566 tablet_bootstrap.cc:492] T adaeaddfb1e94ea1baf46d62657ef3bf P e2b70782d5814ce8a9beb0f8712762c9: Bootstrap starting.
I20260812 07:03:21.833443 26564 tablet_bootstrap.cc:492] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c: Bootstrap starting.
I20260812 07:03:21.833649 26565 tablet_bootstrap.cc:654] T adaeaddfb1e94ea1baf46d62657ef3bf P 162f4d9eeba348f9a9e103b57d8a85ae: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:21.834820 26564 tablet_bootstrap.cc:654] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:21.834923 26566 tablet_bootstrap.cc:654] T adaeaddfb1e94ea1baf46d62657ef3bf P e2b70782d5814ce8a9beb0f8712762c9: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:21.835811 26565 tablet_bootstrap.cc:492] T adaeaddfb1e94ea1baf46d62657ef3bf P 162f4d9eeba348f9a9e103b57d8a85ae: No bootstrap required, opened a new log
I20260812 07:03:21.835986 26565 ts_tablet_manager.cc:1397] T adaeaddfb1e94ea1baf46d62657ef3bf P 162f4d9eeba348f9a9e103b57d8a85ae: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260812 07:03:21.836755 26565 raft_consensus.cc:348] T adaeaddfb1e94ea1baf46d62657ef3bf P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:21.837005 26565 raft_consensus.cc:374] T adaeaddfb1e94ea1baf46d62657ef3bf P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:21.837097 26565 raft_consensus.cc:729] T adaeaddfb1e94ea1baf46d62657ef3bf P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 162f4d9eeba348f9a9e103b57d8a85ae, State: Initialized, Role: FOLLOWER
I20260812 07:03:21.837410 26565 consensus_queue.cc:260] T adaeaddfb1e94ea1baf46d62657ef3bf P 162f4d9eeba348f9a9e103b57d8a85ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:21.837827 26564 tablet_bootstrap.cc:492] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c: No bootstrap required, opened a new log
I20260812 07:03:21.838034 26564 ts_tablet_manager.cc:1397] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 07:03:21.837869 26581 tablet_replica.cc:439] T adaeaddfb1e94ea1baf46d62657ef3bf P 162f4d9eeba348f9a9e103b57d8a85ae: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } } }
I20260812 07:03:21.838181 26581 tablet_replica.cc:442] T adaeaddfb1e94ea1baf46d62657ef3bf P 162f4d9eeba348f9a9e103b57d8a85ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:03:21.838207 26566 tablet_bootstrap.cc:492] T adaeaddfb1e94ea1baf46d62657ef3bf P e2b70782d5814ce8a9beb0f8712762c9: No bootstrap required, opened a new log
I20260812 07:03:21.838366 26566 ts_tablet_manager.cc:1397] T adaeaddfb1e94ea1baf46d62657ef3bf P e2b70782d5814ce8a9beb0f8712762c9: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.003s
I20260812 07:03:21.838928 26565 ts_tablet_manager.cc:1428] T adaeaddfb1e94ea1baf46d62657ef3bf P 162f4d9eeba348f9a9e103b57d8a85ae: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 07:03:21.838918 26564 raft_consensus.cc:348] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:21.839138 26564 raft_consensus.cc:374] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:21.839006 26566 raft_consensus.cc:348] T adaeaddfb1e94ea1baf46d62657ef3bf P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:21.839195 26564 raft_consensus.cc:729] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beb93a68b67948b285012d477c962c4c, State: Initialized, Role: FOLLOWER
I20260812 07:03:21.839232 26566 raft_consensus.cc:374] T adaeaddfb1e94ea1baf46d62657ef3bf P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:21.839303 26566 raft_consensus.cc:729] T adaeaddfb1e94ea1baf46d62657ef3bf P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b70782d5814ce8a9beb0f8712762c9, State: Initialized, Role: FOLLOWER
I20260812 07:03:21.839382 26564 consensus_queue.cc:260] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:21.839557 26566 consensus_queue.cc:260] T adaeaddfb1e94ea1baf46d62657ef3bf P e2b70782d5814ce8a9beb0f8712762c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:21.839789 26575 tablet_replica.cc:439] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } } }
I20260812 07:03:21.840067 26575 tablet_replica.cc:442] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:03:21.839924 26601 tablet_replica.cc:439] T adaeaddfb1e94ea1baf46d62657ef3bf P e2b70782d5814ce8a9beb0f8712762c9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } } }
I20260812 07:03:21.840214 26601 tablet_replica.cc:442] T adaeaddfb1e94ea1baf46d62657ef3bf P e2b70782d5814ce8a9beb0f8712762c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:03:21.841126 26566 ts_tablet_manager.cc:1428] T adaeaddfb1e94ea1baf46d62657ef3bf P e2b70782d5814ce8a9beb0f8712762c9: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260812 07:03:21.841607 26564 ts_tablet_manager.cc:1428] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260812 07:03:21.912734 26575 raft_consensus.cc:482] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:03:21.912905 26575 raft_consensus.cc:504] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:21.914292 26575 leader_election.cc:288] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b70782d5814ce8a9beb0f8712762c9 (127.11.192.193:40691), 162f4d9eeba348f9a9e103b57d8a85ae (127.11.192.194:36263)
I20260812 07:03:21.933629 26634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:21.914680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:60123 (local address 127.11.192.193:40691)
0812 07:03:21.927193 (+ 12513us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:21.927198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:21.927217 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:03:21.927285 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:21.927288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:21.927345 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:21.927443 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:21.928004 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.929077 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.931447 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.932722 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.932893 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:21.932969 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:21.933196 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 07:03:21.933314 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12383,"thread_start_us":12277,"threads_started":1}
I20260812 07:03:21.933749 26638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:21.914685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:48153 (local address 127.11.192.194:36263)
0812 07:03:21.926629 (+ 11944us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:21.926636 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:21.927425 (+ 789us) server_negotiation.cc:407] Connection header received
0812 07:03:21.927590 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:21.927595 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:21.927657 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:21.927732 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:21.928322 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.929378 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.931571 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.932805 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.932945 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:21.933010 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:21.933312 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 07:03:21.933436 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11771,"thread_start_us":11663,"threads_started":1}
I20260812 07:03:21.933791 26633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:21.914569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:36263 (local address 127.11.192.195:48153)
0812 07:03:21.927328 (+ 12759us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:21.927341 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:21.927443 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:21.927796 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:21.927801 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:21.927826 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:21.928176 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:21.928186 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.929492 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.929497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:21.931440 (+ 1943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:21.931451 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.933013 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.933019 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:21.933174 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:21.933202 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 07:03:21.933275 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 07:03:21.933367 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12674,"mutex_wait_us":28,"thread_start_us":12497,"threads_started":1}
I20260812 07:03:21.933935 26627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:21.914559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:40691 (local address 127.11.192.195:60123)
0812 07:03:21.926877 (+ 12318us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:21.926899 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:21.927031 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:21.927481 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:21.927487 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:21.927511 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:21.927796 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:21.927807 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.929231 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.929235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:21.931307 (+ 2072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:21.931324 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.932856 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.932860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:21.933036 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:21.933077 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 07:03:21.933174 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 07:03:21.933301 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12119,"thread_start_us":11984,"threads_started":1}
I20260812 07:03:21.934485 26216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adaeaddfb1e94ea1baf46d62657ef3bf" candidate_uuid: "beb93a68b67948b285012d477c962c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" is_pre_election: true
I20260812 07:03:21.934721 26216 raft_consensus.cc:2454] T adaeaddfb1e94ea1baf46d62657ef3bf P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate beb93a68b67948b285012d477c962c4c in term 0.
I20260812 07:03:21.934834 25962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adaeaddfb1e94ea1baf46d62657ef3bf" candidate_uuid: "beb93a68b67948b285012d477c962c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b70782d5814ce8a9beb0f8712762c9" is_pre_election: true
I20260812 07:03:21.935195 26322 leader_election.cc:302] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 162f4d9eeba348f9a9e103b57d8a85ae, beb93a68b67948b285012d477c962c4c; no voters:
I20260812 07:03:21.935253 25962 raft_consensus.cc:2454] T adaeaddfb1e94ea1baf46d62657ef3bf P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate beb93a68b67948b285012d477c962c4c in term 0.
I20260812 07:03:21.935599 26575 raft_consensus.cc:2781] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:03:21.935722 26575 raft_consensus.cc:482] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:03:21.935827 26575 raft_consensus.cc:3037] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:21.937871 26575 raft_consensus.cc:504] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:21.938407 26575 leader_election.cc:288] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c [CANDIDATE]: Term 1 election: Requested vote from peers e2b70782d5814ce8a9beb0f8712762c9 (127.11.192.193:40691), 162f4d9eeba348f9a9e103b57d8a85ae (127.11.192.194:36263)
I20260812 07:03:21.938906 26216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adaeaddfb1e94ea1baf46d62657ef3bf" candidate_uuid: "beb93a68b67948b285012d477c962c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "162f4d9eeba348f9a9e103b57d8a85ae"
I20260812 07:03:21.938922 25962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adaeaddfb1e94ea1baf46d62657ef3bf" candidate_uuid: "beb93a68b67948b285012d477c962c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b70782d5814ce8a9beb0f8712762c9"
I20260812 07:03:21.939113 25962 raft_consensus.cc:3037] T adaeaddfb1e94ea1baf46d62657ef3bf P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:21.939126 26216 raft_consensus.cc:3037] T adaeaddfb1e94ea1baf46d62657ef3bf P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:21.941890 25962 raft_consensus.cc:2454] T adaeaddfb1e94ea1baf46d62657ef3bf P e2b70782d5814ce8a9beb0f8712762c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate beb93a68b67948b285012d477c962c4c in term 1.
I20260812 07:03:21.941989 26216 raft_consensus.cc:2454] T adaeaddfb1e94ea1baf46d62657ef3bf P 162f4d9eeba348f9a9e103b57d8a85ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate beb93a68b67948b285012d477c962c4c in term 1.
I20260812 07:03:21.942355 26315 leader_election.cc:302] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: beb93a68b67948b285012d477c962c4c, e2b70782d5814ce8a9beb0f8712762c9; no voters:
I20260812 07:03:21.942627 26575 raft_consensus.cc:2781] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:03:21.942971 26575 raft_consensus.cc:686] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c [term 1 LEADER]: Becoming Leader. State: Replica: beb93a68b67948b285012d477c962c4c, State: Running, Role: LEADER
I20260812 07:03:21.943369 26575 consensus_queue.cc:237] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:21.945200 26640 tablet_replica.cc:439] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c: TxnStatusTablet state changed. Reason: New leader beb93a68b67948b285012d477c962c4c. Latest consensus state: current_term: 1 leader_uuid: "beb93a68b67948b285012d477c962c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } } }
I20260812 07:03:21.945580 26640 tablet_replica.cc:442] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:03:21.946185 26646 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 07:03:21.946385 26646 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 07:03:21.946892 25695 catalog_manager.cc:5495] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c reported cstate change: term changed from 0 to 1, leader changed from <none> to beb93a68b67948b285012d477c962c4c (127.11.192.195). New cstate: current_term: 1 leader_uuid: "beb93a68b67948b285012d477c962c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } health_report { overall_health: HEALTHY } } }
I20260812 07:03:21.990852 26623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:21.985013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:34345 (local address 127.0.0.1:54370)
0812 07:03:21.985232 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:21.985248 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:21.985385 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:21.986069 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:21.986075 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:21.986088 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:03:21.986288 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:21.986299 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.987414 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.987420 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:21.988433 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:21.988446 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.989178 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.989184 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:21.989335 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:21.989388 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:03:21.990287 (+ 899us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:03:21.990310 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 07:03:21.990389 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 07:03:21.990540 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 07:03:21.990880 26652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:21.985106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54370 (local address 127.11.192.195:34345)
0812 07:03:21.985697 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:21.985703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:21.985724 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:03:21.985800 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:21.985804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:21.985879 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:21.986020 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:03:21.986434 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.987293 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.988572 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:21.989027 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:21.989139 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:21.989595 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:03:21.990182 (+ 587us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:03:21.990318 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:21.990423 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 07:03:21.990504 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":282,"threads_started":1}
I20260812 07:03:21.995225 26216 raft_consensus.cc:1261] T adaeaddfb1e94ea1baf46d62657ef3bf P 162f4d9eeba348f9a9e103b57d8a85ae [term 1 FOLLOWER]: Refusing update from remote peer beb93a68b67948b285012d477c962c4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:03:21.995368 25962 raft_consensus.cc:1261] T adaeaddfb1e94ea1baf46d62657ef3bf P e2b70782d5814ce8a9beb0f8712762c9 [term 1 FOLLOWER]: Refusing update from remote peer beb93a68b67948b285012d477c962c4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:03:21.995961 26640 consensus_queue.cc:1035] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:03:21.996655 26575 consensus_queue.cc:1035] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:03:22.000872 26601 tablet_replica.cc:439] T adaeaddfb1e94ea1baf46d62657ef3bf P e2b70782d5814ce8a9beb0f8712762c9: TxnStatusTablet state changed. Reason: New leader beb93a68b67948b285012d477c962c4c. Latest consensus state: current_term: 1 leader_uuid: "beb93a68b67948b285012d477c962c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } } }
I20260812 07:03:22.001159 26601 tablet_replica.cc:442] T adaeaddfb1e94ea1baf46d62657ef3bf P e2b70782d5814ce8a9beb0f8712762c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:03:22.005592 26581 tablet_replica.cc:439] T adaeaddfb1e94ea1baf46d62657ef3bf P 162f4d9eeba348f9a9e103b57d8a85ae: TxnStatusTablet state changed. Reason: New leader beb93a68b67948b285012d477c962c4c. Latest consensus state: current_term: 1 leader_uuid: "beb93a68b67948b285012d477c962c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } } }
I20260812 07:03:22.005890 26581 tablet_replica.cc:442] T adaeaddfb1e94ea1baf46d62657ef3bf P 162f4d9eeba348f9a9e103b57d8a85ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:03:22.006098 26655 mvcc.cc:205] Tried to move back new op lower bound from 7317578555364904960 to 7317578555163566080. Current Snapshot: MvccSnapshot[applied={T|T < 7317578555364904960}]
I20260812 07:03:22.006247 26601 tablet_replica.cc:439] T adaeaddfb1e94ea1baf46d62657ef3bf P e2b70782d5814ce8a9beb0f8712762c9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "beb93a68b67948b285012d477c962c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } } }
I20260812 07:03:22.006508 26601 tablet_replica.cc:442] T adaeaddfb1e94ea1baf46d62657ef3bf P e2b70782d5814ce8a9beb0f8712762c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:03:22.015745 26581 tablet_replica.cc:439] T adaeaddfb1e94ea1baf46d62657ef3bf P 162f4d9eeba348f9a9e103b57d8a85ae: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "beb93a68b67948b285012d477c962c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } } }
I20260812 07:03:22.016010 26581 tablet_replica.cc:442] T adaeaddfb1e94ea1baf46d62657ef3bf P 162f4d9eeba348f9a9e103b57d8a85ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:03:22.016796 26575 tablet_replica.cc:439] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "beb93a68b67948b285012d477c962c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } } }
I20260812 07:03:22.017217 26575 tablet_replica.cc:442] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:03:22.018227 26575 tablet_replica.cc:439] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "beb93a68b67948b285012d477c962c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } } }
I20260812 07:03:22.018793 26575 tablet_replica.cc:442] T adaeaddfb1e94ea1baf46d62657ef3bf P beb93a68b67948b285012d477c962c4c: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:03:22.041632 26652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:22.036327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:60147 (local address 127.11.192.195:34345)
0812 07:03:22.036469 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:22.036474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:22.036944 (+ 470us) server_negotiation.cc:407] Connection header received
0812 07:03:22.037138 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:22.037143 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:22.037210 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:22.037284 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:03:22.037769 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.038493 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.039816 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.040328 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.040485 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:22.040867 (+ 382us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:03:22.040969 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:03:22.041047 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:22.041262 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 07:03:22.041355 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 07:03:22.041714 26676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:22.036276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:34345 (local address 127.11.192.193:60147)
0812 07:03:22.036845 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:22.036860 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:22.037005 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:22.037314 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:22.037319 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:22.037331 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:03:22.037615 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:22.037624 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.038637 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.038642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:22.039677 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:22.039689 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.040516 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.040522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:22.040656 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:22.040707 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:03:22.041066 (+ 359us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:03:22.041078 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 07:03:22.041208 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 07:03:22.041351 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":327,"threads_started":1}
I20260812 07:03:22.051688 26440 raft_consensus.cc:1261] T 3bd2f0fc398f44248483fc239933b201 P beb93a68b67948b285012d477c962c4c [term 1 FOLLOWER]: Refusing update from remote peer e2b70782d5814ce8a9beb0f8712762c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:03:22.052234 26216 raft_consensus.cc:1261] T 3bd2f0fc398f44248483fc239933b201 P 162f4d9eeba348f9a9e103b57d8a85ae [term 1 FOLLOWER]: Refusing update from remote peer e2b70782d5814ce8a9beb0f8712762c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:03:22.052210 26601 consensus_queue.cc:1035] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:03:22.052779 26601 consensus_queue.cc:1035] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:03:22.093266 26634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:22.086309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:60071 (local address 127.11.192.193:40691)
0812 07:03:22.086655 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:22.086662 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:22.087528 (+ 866us) server_negotiation.cc:407] Connection header received
0812 07:03:22.087903 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:22.087909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:22.087974 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:22.088048 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:03:22.088607 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.089287 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.091209 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.091851 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.091993 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:22.092575 (+ 582us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:03:22.092706 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:03:22.092801 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:22.092906 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 07:03:22.092997 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218}
I20260812 07:03:22.093344 26687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:22.085890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:40691 (local address 127.11.192.195:60071)
0812 07:03:22.087430 (+ 1540us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:22.087455 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:22.087775 (+ 320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:22.088101 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:22.088105 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:22.088117 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:03:22.088445 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:22.088454 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.089415 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.089420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:22.090725 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:22.090740 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.092191 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.092198 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:22.092329 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:22.092399 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:03:22.092800 (+ 401us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:03:22.092809 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 07:03:22.092867 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 07:03:22.092971 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1097,"thread_start_us":933,"threads_started":1}
I20260812 07:03:22.124673 25693 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:03:22.124938 25693 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:03:22.127861 25693 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883: Sending DeleteTablet for 3 replicas of tablet 3bd2f0fc398f44248483fc239933b201
I20260812 07:03:22.128538 25932 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bd2f0fc398f44248483fc239933b201 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:03:22 UTC) from {username='oryx', principal='oryx/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:56944
I20260812 07:03:22.128728 26165 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bd2f0fc398f44248483fc239933b201 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:03:22 UTC) from {username='oryx', principal='oryx/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:45256
I20260812 07:03:22.129045 26410 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bd2f0fc398f44248483fc239933b201 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:03:22 UTC) from {username='oryx', principal='oryx/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:54354
I20260812 07:03:22.153439 26691 tablet_replica.cc:330] T 3bd2f0fc398f44248483fc239933b201 P 162f4d9eeba348f9a9e103b57d8a85ae: stopping tablet replica
I20260812 07:03:22.153954 26691 raft_consensus.cc:2229] T 3bd2f0fc398f44248483fc239933b201 P 162f4d9eeba348f9a9e103b57d8a85ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:03:22.154404 26691 raft_consensus.cc:2258] T 3bd2f0fc398f44248483fc239933b201 P 162f4d9eeba348f9a9e103b57d8a85ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:03:22.155267 26695 tablet_replica.cc:330] T 3bd2f0fc398f44248483fc239933b201 P beb93a68b67948b285012d477c962c4c: stopping tablet replica
I20260812 07:03:22.155568 26695 raft_consensus.cc:2229] T 3bd2f0fc398f44248483fc239933b201 P beb93a68b67948b285012d477c962c4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:03:22.155938 26691 ts_tablet_manager.cc:1905] T 3bd2f0fc398f44248483fc239933b201 P 162f4d9eeba348f9a9e103b57d8a85ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:03:22.155957 26695 raft_consensus.cc:2258] T 3bd2f0fc398f44248483fc239933b201 P beb93a68b67948b285012d477c962c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:03:22.157496 26695 ts_tablet_manager.cc:1905] T 3bd2f0fc398f44248483fc239933b201 P beb93a68b67948b285012d477c962c4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:03:22.163347 26691 ts_tablet_manager.cc:1918] T 3bd2f0fc398f44248483fc239933b201 P 162f4d9eeba348f9a9e103b57d8a85ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:03:22.163494 26691 log.cc:1196] T 3bd2f0fc398f44248483fc239933b201 P 162f4d9eeba348f9a9e103b57d8a85ae: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/3bd2f0fc398f44248483fc239933b201
I20260812 07:03:22.163970 26691 ts_tablet_manager.cc:1939] T 3bd2f0fc398f44248483fc239933b201 P 162f4d9eeba348f9a9e103b57d8a85ae: Deleting consensus metadata
I20260812 07:03:22.165165 25667 catalog_manager.cc:4841] TS 162f4d9eeba348f9a9e103b57d8a85ae (127.11.192.194:36263): tablet 3bd2f0fc398f44248483fc239933b201 (table test-table [id=79b465a35fc8495d84312b33f436c29c]) successfully deleted
I20260812 07:03:22.165889 26689 tablet_replica.cc:330] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9: stopping tablet replica
I20260812 07:03:22.166101 26689 raft_consensus.cc:2229] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:03:22.166631 26689 raft_consensus.cc:2258] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:03:22.167757 26689 ts_tablet_manager.cc:1905] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:03:22.172463 26695 ts_tablet_manager.cc:1918] T 3bd2f0fc398f44248483fc239933b201 P beb93a68b67948b285012d477c962c4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:03:22.172636 26695 log.cc:1196] T 3bd2f0fc398f44248483fc239933b201 P beb93a68b67948b285012d477c962c4c: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/3bd2f0fc398f44248483fc239933b201
I20260812 07:03:22.173139 26695 ts_tablet_manager.cc:1939] T 3bd2f0fc398f44248483fc239933b201 P beb93a68b67948b285012d477c962c4c: Deleting consensus metadata
I20260812 07:03:22.174283 25665 catalog_manager.cc:4841] TS beb93a68b67948b285012d477c962c4c (127.11.192.195:34345): tablet 3bd2f0fc398f44248483fc239933b201 (table test-table [id=79b465a35fc8495d84312b33f436c29c]) successfully deleted
I20260812 07:03:22.175112 26689 ts_tablet_manager.cc:1918] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:03:22.175371 26689 log.cc:1196] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-0/wal/wals/3bd2f0fc398f44248483fc239933b201
I20260812 07:03:22.175915 26689 ts_tablet_manager.cc:1939] T 3bd2f0fc398f44248483fc239933b201 P e2b70782d5814ce8a9beb0f8712762c9: Deleting consensus metadata
I20260812 07:03:22.177138 25663 catalog_manager.cc:4841] TS e2b70782d5814ce8a9beb0f8712762c9 (127.11.192.193:40691): tablet 3bd2f0fc398f44248483fc239933b201 (table test-table [id=79b465a35fc8495d84312b33f436c29c]) successfully deleted
Aug 12 07:03:22 dist-test-slave-lgr5 krb5kdc[25605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 07:03:22 dist-test-slave-lgr5 krb5kdc[25605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:22 dist-test-slave-lgr5 krb5kdc[25605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 07:03:22.218254 26622 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:22.206558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53248 (local address 127.11.192.254:43629)
0812 07:03:22.206732 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:22.206737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:22.207169 (+ 432us) server_negotiation.cc:407] Connection header received
0812 07:03:22.207386 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:22.207390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:22.207454 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:22.207534 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:22.209432 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.210168 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.212016 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.212475 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.212569 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:22.218005 (+ 5436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53248: BlockingRecv error: recv got EOF from 127.0.0.1:53248 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 12 07:03:22 dist-test-slave-lgr5 krb5kdc[25605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM
I20260812 07:03:22.235863 26622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:22.221607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53254 (local address 127.11.192.254:43629)
0812 07:03:22.221785 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:22.221791 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:22.222145 (+ 354us) server_negotiation.cc:407] Connection header received
0812 07:03:22.222263 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:22.222269 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:22.222354 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:22.222464 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:22.223478 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.224616 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.225826 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.226505 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.226647 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:22.230557 (+ 3910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:22.230562 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:22.230565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:22.230596 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:22.233194 (+ 2598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:22.233983 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:22.233986 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:22.233988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:22.234110 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:22.234514 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:22.234517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:22.234519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:22.234775 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:22.235056 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:22.235333 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 07:03:22.235559 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:03:22.239414 25693 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53254:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:03:22.239979 25693 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:03:22.246965 25932 tablet_service.cc:1467] Processing CreateTablet for tablet 1b8c67b5538a4e63abe150eabf5fb0a2 (DEFAULT_TABLE table=test-table [id=27679eeba4594725883933107bf4f27c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:03:22.247427 25932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b8c67b5538a4e63abe150eabf5fb0a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:22.247447 26165 tablet_service.cc:1467] Processing CreateTablet for tablet 1b8c67b5538a4e63abe150eabf5fb0a2 (DEFAULT_TABLE table=test-table [id=27679eeba4594725883933107bf4f27c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:03:22.247694 26410 tablet_service.cc:1467] Processing CreateTablet for tablet 1b8c67b5538a4e63abe150eabf5fb0a2 (DEFAULT_TABLE table=test-table [id=27679eeba4594725883933107bf4f27c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:03:22.247882 26165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b8c67b5538a4e63abe150eabf5fb0a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:22.248240 26410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b8c67b5538a4e63abe150eabf5fb0a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:22.253772 26566 tablet_bootstrap.cc:492] T 1b8c67b5538a4e63abe150eabf5fb0a2 P e2b70782d5814ce8a9beb0f8712762c9: Bootstrap starting.
I20260812 07:03:22.254057 26565 tablet_bootstrap.cc:492] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae: Bootstrap starting.
I20260812 07:03:22.255003 26566 tablet_bootstrap.cc:654] T 1b8c67b5538a4e63abe150eabf5fb0a2 P e2b70782d5814ce8a9beb0f8712762c9: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:22.255793 26565 tablet_bootstrap.cc:654] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:22.256897 26566 tablet_bootstrap.cc:492] T 1b8c67b5538a4e63abe150eabf5fb0a2 P e2b70782d5814ce8a9beb0f8712762c9: No bootstrap required, opened a new log
I20260812 07:03:22.257028 26566 ts_tablet_manager.cc:1397] T 1b8c67b5538a4e63abe150eabf5fb0a2 P e2b70782d5814ce8a9beb0f8712762c9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 07:03:22.257598 26566 raft_consensus.cc:348] T 1b8c67b5538a4e63abe150eabf5fb0a2 P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:22.257855 26566 raft_consensus.cc:374] T 1b8c67b5538a4e63abe150eabf5fb0a2 P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:22.257956 26566 raft_consensus.cc:729] T 1b8c67b5538a4e63abe150eabf5fb0a2 P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b70782d5814ce8a9beb0f8712762c9, State: Initialized, Role: FOLLOWER
I20260812 07:03:22.258179 26566 consensus_queue.cc:260] T 1b8c67b5538a4e63abe150eabf5fb0a2 P e2b70782d5814ce8a9beb0f8712762c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:22.258659 26566 ts_tablet_manager.cc:1428] T 1b8c67b5538a4e63abe150eabf5fb0a2 P e2b70782d5814ce8a9beb0f8712762c9: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260812 07:03:22.258806 26565 tablet_bootstrap.cc:492] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae: No bootstrap required, opened a new log
I20260812 07:03:22.258949 26565 ts_tablet_manager.cc:1397] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260812 07:03:22.260649 26565 raft_consensus.cc:348] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:22.260988 26565 raft_consensus.cc:374] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:22.261130 26565 raft_consensus.cc:729] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 162f4d9eeba348f9a9e103b57d8a85ae, State: Initialized, Role: FOLLOWER
I20260812 07:03:22.261427 26565 consensus_queue.cc:260] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:22.262063 26565 ts_tablet_manager.cc:1428] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260812 07:03:22.264534 26564 tablet_bootstrap.cc:492] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c: Bootstrap starting.
I20260812 07:03:22.266350 26564 tablet_bootstrap.cc:654] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:22.268704 26564 tablet_bootstrap.cc:492] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c: No bootstrap required, opened a new log
I20260812 07:03:22.268859 26564 ts_tablet_manager.cc:1397] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260812 07:03:22.269739 26564 raft_consensus.cc:348] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:22.270020 26564 raft_consensus.cc:374] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:22.270125 26564 raft_consensus.cc:729] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beb93a68b67948b285012d477c962c4c, State: Initialized, Role: FOLLOWER
I20260812 07:03:22.270409 26564 consensus_queue.cc:260] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:22.271662 26564 ts_tablet_manager.cc:1428] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 07:03:22.371943 26581 raft_consensus.cc:482] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:03:22.372207 26581 raft_consensus.cc:504] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:22.373678 26581 leader_election.cc:288] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b70782d5814ce8a9beb0f8712762c9 (127.11.192.193:40691), beb93a68b67948b285012d477c962c4c (127.11.192.195:34345)
I20260812 07:03:22.379418 26575 raft_consensus.cc:482] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:03:22.379607 26575 raft_consensus.cc:504] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:22.380405 26575 leader_election.cc:288] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b70782d5814ce8a9beb0f8712762c9 (127.11.192.193:40691), 162f4d9eeba348f9a9e103b57d8a85ae (127.11.192.194:36263)
I20260812 07:03:22.381239 26216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b8c67b5538a4e63abe150eabf5fb0a2" candidate_uuid: "beb93a68b67948b285012d477c962c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" is_pre_election: true
I20260812 07:03:22.381542 26216 raft_consensus.cc:2454] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate beb93a68b67948b285012d477c962c4c in term 0.
I20260812 07:03:22.382007 26634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:22.374894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:51241 (local address 127.11.192.193:40691)
0812 07:03:22.375097 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:22.375103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:22.375128 (+ 25us) server_negotiation.cc:407] Connection header received
0812 07:03:22.375208 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:22.375214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:22.375300 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:22.375398 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:22.376167 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.376897 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.379536 (+ 2639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.380762 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.380896 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:22.380973 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:22.381583 (+ 610us) server_negotiation.cc:299] Negotiation successful
0812 07:03:22.381725 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 07:03:22.382059 26322 leader_election.cc:302] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 162f4d9eeba348f9a9e103b57d8a85ae, beb93a68b67948b285012d477c962c4c; no voters:
I20260812 07:03:22.382715 26725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:22.374073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:40691 (local address 127.11.192.194:51241)
0812 07:03:22.374951 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:22.374967 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:22.375081 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:22.375501 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:22.375509 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:22.375551 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:22.375973 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:22.375985 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.377032 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.377038 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:22.379327 (+ 2289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:22.379342 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.381187 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.381194 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:22.381381 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:22.381411 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 07:03:22.382184 (+ 773us) client_negotiation.cc:240] Negotiation successful
0812 07:03:22.382317 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"mutex_wait_us":72,"spinlock_wait_cycles":896,"thread_start_us":1090,"threads_started":1}
I20260812 07:03:22.382715 26652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:22.374137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:34249 (local address 127.11.192.195:34345)
0812 07:03:22.374662 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:22.374670 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:22.374695 (+ 25us) server_negotiation.cc:407] Connection header received
0812 07:03:22.374936 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:22.374942 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:22.375037 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:22.375115 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:22.376073 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.377256 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.379792 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.381132 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.381252 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:22.381319 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:22.382334 (+ 1015us) server_negotiation.cc:299] Negotiation successful
0812 07:03:22.382469 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353}
I20260812 07:03:22.382738 25962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b8c67b5538a4e63abe150eabf5fb0a2" candidate_uuid: "beb93a68b67948b285012d477c962c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b70782d5814ce8a9beb0f8712762c9" is_pre_election: true
I20260812 07:03:22.383005 25962 raft_consensus.cc:2454] T 1b8c67b5538a4e63abe150eabf5fb0a2 P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate beb93a68b67948b285012d477c962c4c in term 0.
I20260812 07:03:22.383592 26723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:22.373966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:34345 (local address 127.11.192.194:34249)
0812 07:03:22.374567 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:22.374589 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:22.374733 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:22.375336 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:22.375340 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:22.375378 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:22.375826 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:22.375842 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.377406 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.377411 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:22.379514 (+ 2103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:22.379630 (+ 116us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.381749 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.381758 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:22.381936 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:22.381979 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 07:03:22.383159 (+ 1180us) client_negotiation.cc:240] Negotiation successful
0812 07:03:22.383270 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":18,"spinlock_wait_cycles":39552,"thread_start_us":350,"threads_started":1}
I20260812 07:03:22.383754 25962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b8c67b5538a4e63abe150eabf5fb0a2" candidate_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b70782d5814ce8a9beb0f8712762c9" is_pre_election: true
I20260812 07:03:22.383973 25962 raft_consensus.cc:2454] T 1b8c67b5538a4e63abe150eabf5fb0a2 P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 162f4d9eeba348f9a9e103b57d8a85ae in term 0.
I20260812 07:03:22.384159 26440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b8c67b5538a4e63abe150eabf5fb0a2" candidate_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "beb93a68b67948b285012d477c962c4c" is_pre_election: true
I20260812 07:03:22.384176 26575 raft_consensus.cc:2781] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:03:22.384588 26575 raft_consensus.cc:482] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:03:22.384505 26088 leader_election.cc:302] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 162f4d9eeba348f9a9e103b57d8a85ae, e2b70782d5814ce8a9beb0f8712762c9; no voters:
I20260812 07:03:22.384681 26575 raft_consensus.cc:3037] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:22.384919 26581 raft_consensus.cc:2781] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:03:22.385035 26581 raft_consensus.cc:482] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:03:22.385115 26581 raft_consensus.cc:3037] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:22.387152 26575 raft_consensus.cc:504] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:22.387547 26440 raft_consensus.cc:2379] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 162f4d9eeba348f9a9e103b57d8a85ae in current term 1: Already voted for candidate beb93a68b67948b285012d477c962c4c in this term.
I20260812 07:03:22.388234 26575 leader_election.cc:288] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [CANDIDATE]: Term 1 election: Requested vote from peers e2b70782d5814ce8a9beb0f8712762c9 (127.11.192.193:40691), 162f4d9eeba348f9a9e103b57d8a85ae (127.11.192.194:36263)
I20260812 07:03:22.388253 26581 raft_consensus.cc:504] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:22.388839 26216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b8c67b5538a4e63abe150eabf5fb0a2" candidate_uuid: "beb93a68b67948b285012d477c962c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "162f4d9eeba348f9a9e103b57d8a85ae"
I20260812 07:03:22.388757 25962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b8c67b5538a4e63abe150eabf5fb0a2" candidate_uuid: "beb93a68b67948b285012d477c962c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b70782d5814ce8a9beb0f8712762c9"
I20260812 07:03:22.389117 25962 raft_consensus.cc:3037] T 1b8c67b5538a4e63abe150eabf5fb0a2 P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:22.389201 26216 raft_consensus.cc:2379] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate beb93a68b67948b285012d477c962c4c in current term 1: Already voted for candidate 162f4d9eeba348f9a9e103b57d8a85ae in this term.
I20260812 07:03:22.390268 25961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b8c67b5538a4e63abe150eabf5fb0a2" candidate_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b70782d5814ce8a9beb0f8712762c9"
I20260812 07:03:22.390417 26440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b8c67b5538a4e63abe150eabf5fb0a2" candidate_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "beb93a68b67948b285012d477c962c4c"
I20260812 07:03:22.390702 26440 raft_consensus.cc:2379] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 162f4d9eeba348f9a9e103b57d8a85ae in current term 1: Already voted for candidate beb93a68b67948b285012d477c962c4c in this term.
I20260812 07:03:22.391202 26089 leader_election.cc:302] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 162f4d9eeba348f9a9e103b57d8a85ae; no voters: beb93a68b67948b285012d477c962c4c, e2b70782d5814ce8a9beb0f8712762c9
I20260812 07:03:22.391748 25962 raft_consensus.cc:2454] T 1b8c67b5538a4e63abe150eabf5fb0a2 P e2b70782d5814ce8a9beb0f8712762c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate beb93a68b67948b285012d477c962c4c in term 1.
I20260812 07:03:22.391887 26729 raft_consensus.cc:2726] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260812 07:03:22.392313 26315 leader_election.cc:302] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: beb93a68b67948b285012d477c962c4c, e2b70782d5814ce8a9beb0f8712762c9; no voters: 162f4d9eeba348f9a9e103b57d8a85ae
I20260812 07:03:22.392688 26581 leader_election.cc:288] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae [CANDIDATE]: Term 1 election: Requested vote from peers e2b70782d5814ce8a9beb0f8712762c9 (127.11.192.193:40691), beb93a68b67948b285012d477c962c4c (127.11.192.195:34345)
I20260812 07:03:22.392985 26575 raft_consensus.cc:2781] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:03:22.393143 26575 raft_consensus.cc:686] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [term 1 LEADER]: Becoming Leader. State: Replica: beb93a68b67948b285012d477c962c4c, State: Running, Role: LEADER
I20260812 07:03:22.393412 26575 consensus_queue.cc:237] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:22.396190 25695 catalog_manager.cc:5495] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c reported cstate change: term changed from 0 to 1, leader changed from <none> to beb93a68b67948b285012d477c962c4c (127.11.192.195). New cstate: current_term: 1 leader_uuid: "beb93a68b67948b285012d477c962c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } health_report { overall_health: HEALTHY } } }
I20260812 07:03:22.427184 26652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:22.422812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54382 (local address 127.11.192.195:34345)
0812 07:03:22.422967 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:22.422978 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:22.423051 (+ 73us) server_negotiation.cc:407] Connection header received
0812 07:03:22.423199 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:22.423203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:22.423263 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:22.423335 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:03:22.423635 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.424418 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.425738 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.426147 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.426234 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:22.426595 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:03:22.426692 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:03:22.426753 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:22.426872 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 07:03:22.426929 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:03:22.430084 25962 raft_consensus.cc:1261] T 1b8c67b5538a4e63abe150eabf5fb0a2 P e2b70782d5814ce8a9beb0f8712762c9 [term 1 FOLLOWER]: Refusing update from remote peer beb93a68b67948b285012d477c962c4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:03:22.430116 26216 raft_consensus.cc:1261] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae [term 1 FOLLOWER]: Refusing update from remote peer beb93a68b67948b285012d477c962c4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:03:22.430632 26575 consensus_queue.cc:1035] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:03:22.430903 26669 consensus_queue.cc:1035] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:03:22.436420 26656 mvcc.cc:205] Tried to move back new op lower bound from 7317578557148426240 to 7317578557005983744. Current Snapshot: MvccSnapshot[applied={T|T < 7317578557148426240}]
I20260812 07:03:22.444847 25695 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:03:22.445043 25695 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:03:22.456388 25695 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883: Sending DeleteTablet for 3 replicas of tablet 1b8c67b5538a4e63abe150eabf5fb0a2
I20260812 07:03:22.457204 25932 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b8c67b5538a4e63abe150eabf5fb0a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:03:22 UTC) from {username='oryx', principal='oryx/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:56944
I20260812 07:03:22.457387 26165 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b8c67b5538a4e63abe150eabf5fb0a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:03:22 UTC) from {username='oryx', principal='oryx/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:45256
I20260812 07:03:22.457682 26689 tablet_replica.cc:330] T 1b8c67b5538a4e63abe150eabf5fb0a2 P e2b70782d5814ce8a9beb0f8712762c9: stopping tablet replica
I20260812 07:03:22.457890 26689 raft_consensus.cc:2229] T 1b8c67b5538a4e63abe150eabf5fb0a2 P e2b70782d5814ce8a9beb0f8712762c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:03:22.458083 26689 raft_consensus.cc:2258] T 1b8c67b5538a4e63abe150eabf5fb0a2 P e2b70782d5814ce8a9beb0f8712762c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:03:22.458309 26691 tablet_replica.cc:330] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae: stopping tablet replica
I20260812 07:03:22.458134 25695 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53254:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 07:03:22.458510 26691 raft_consensus.cc:2229] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:03:22.458730 26691 raft_consensus.cc:2258] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 07:03:22.458756 25695 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:03:22.459245 26689 ts_tablet_manager.cc:1905] T 1b8c67b5538a4e63abe150eabf5fb0a2 P e2b70782d5814ce8a9beb0f8712762c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:03:22.459697 26410 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b8c67b5538a4e63abe150eabf5fb0a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:03:22 UTC) from {username='oryx', principal='oryx/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:54354
I20260812 07:03:22.459951 26695 tablet_replica.cc:330] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c: stopping tablet replica
I20260812 07:03:22.460151 26695 raft_consensus.cc:2229] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:03:22.460614 26695 raft_consensus.cc:2258] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:03:22.461908 26691 ts_tablet_manager.cc:1905] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:03:22.464826 26695 ts_tablet_manager.cc:1905] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:03:22.465523 26689 ts_tablet_manager.cc:1918] T 1b8c67b5538a4e63abe150eabf5fb0a2 P e2b70782d5814ce8a9beb0f8712762c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:03:22.465660 26689 log.cc:1196] T 1b8c67b5538a4e63abe150eabf5fb0a2 P e2b70782d5814ce8a9beb0f8712762c9: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-0/wal/wals/1b8c67b5538a4e63abe150eabf5fb0a2
I20260812 07:03:22.466044 26689 ts_tablet_manager.cc:1939] T 1b8c67b5538a4e63abe150eabf5fb0a2 P e2b70782d5814ce8a9beb0f8712762c9: Deleting consensus metadata
I20260812 07:03:22.467926 26691 ts_tablet_manager.cc:1918] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:03:22.468178 26691 log.cc:1196] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/1b8c67b5538a4e63abe150eabf5fb0a2
I20260812 07:03:22.468274 25663 catalog_manager.cc:4841] TS e2b70782d5814ce8a9beb0f8712762c9 (127.11.192.193:40691): tablet 1b8c67b5538a4e63abe150eabf5fb0a2 (table test-table [id=27679eeba4594725883933107bf4f27c]) successfully deleted
I20260812 07:03:22.468704 26691 ts_tablet_manager.cc:1939] T 1b8c67b5538a4e63abe150eabf5fb0a2 P 162f4d9eeba348f9a9e103b57d8a85ae: Deleting consensus metadata
I20260812 07:03:22.469772 25667 catalog_manager.cc:4841] TS 162f4d9eeba348f9a9e103b57d8a85ae (127.11.192.194:36263): tablet 1b8c67b5538a4e63abe150eabf5fb0a2 (table test-table [id=27679eeba4594725883933107bf4f27c]) successfully deleted
I20260812 07:03:22.471508 26695 ts_tablet_manager.cc:1918] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:03:22.471660 26695 log.cc:1196] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/1b8c67b5538a4e63abe150eabf5fb0a2
I20260812 07:03:22.472169 26695 ts_tablet_manager.cc:1939] T 1b8c67b5538a4e63abe150eabf5fb0a2 P beb93a68b67948b285012d477c962c4c: Deleting consensus metadata
I20260812 07:03:22.472465 25932 tablet_service.cc:1467] Processing CreateTablet for tablet 335bb234196b46b7aa570e6669892348 (DEFAULT_TABLE table=test-table [id=6d222719e7de432ab783d7511e1a8adf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:03:22.472858 25932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335bb234196b46b7aa570e6669892348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:22.474202 25665 catalog_manager.cc:4841] TS beb93a68b67948b285012d477c962c4c (127.11.192.195:34345): tablet 1b8c67b5538a4e63abe150eabf5fb0a2 (table test-table [id=27679eeba4594725883933107bf4f27c]) successfully deleted
I20260812 07:03:22.474913 26410 tablet_service.cc:1467] Processing CreateTablet for tablet 335bb234196b46b7aa570e6669892348 (DEFAULT_TABLE table=test-table [id=6d222719e7de432ab783d7511e1a8adf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:03:22.475322 26410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335bb234196b46b7aa570e6669892348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:22.475363 26165 tablet_service.cc:1467] Processing CreateTablet for tablet 335bb234196b46b7aa570e6669892348 (DEFAULT_TABLE table=test-table [id=6d222719e7de432ab783d7511e1a8adf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:03:22.475756 26165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335bb234196b46b7aa570e6669892348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:22.479112 26566 tablet_bootstrap.cc:492] T 335bb234196b46b7aa570e6669892348 P e2b70782d5814ce8a9beb0f8712762c9: Bootstrap starting.
I20260812 07:03:22.480679 26566 tablet_bootstrap.cc:654] T 335bb234196b46b7aa570e6669892348 P e2b70782d5814ce8a9beb0f8712762c9: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:22.481696 26565 tablet_bootstrap.cc:492] T 335bb234196b46b7aa570e6669892348 P 162f4d9eeba348f9a9e103b57d8a85ae: Bootstrap starting.
I20260812 07:03:22.482949 26564 tablet_bootstrap.cc:492] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c: Bootstrap starting.
I20260812 07:03:22.483165 26565 tablet_bootstrap.cc:654] T 335bb234196b46b7aa570e6669892348 P 162f4d9eeba348f9a9e103b57d8a85ae: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:22.484597 26564 tablet_bootstrap.cc:654] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:22.484929 26565 tablet_bootstrap.cc:492] T 335bb234196b46b7aa570e6669892348 P 162f4d9eeba348f9a9e103b57d8a85ae: No bootstrap required, opened a new log
I20260812 07:03:22.485145 26565 ts_tablet_manager.cc:1397] T 335bb234196b46b7aa570e6669892348 P 162f4d9eeba348f9a9e103b57d8a85ae: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 07:03:22.485867 26565 raft_consensus.cc:348] T 335bb234196b46b7aa570e6669892348 P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:22.486071 26565 raft_consensus.cc:374] T 335bb234196b46b7aa570e6669892348 P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:22.486140 26565 raft_consensus.cc:729] T 335bb234196b46b7aa570e6669892348 P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 162f4d9eeba348f9a9e103b57d8a85ae, State: Initialized, Role: FOLLOWER
I20260812 07:03:22.486455 26565 consensus_queue.cc:260] T 335bb234196b46b7aa570e6669892348 P 162f4d9eeba348f9a9e103b57d8a85ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:22.487018 26565 ts_tablet_manager.cc:1428] T 335bb234196b46b7aa570e6669892348 P 162f4d9eeba348f9a9e103b57d8a85ae: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260812 07:03:22.488653 26564 tablet_bootstrap.cc:492] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c: No bootstrap required, opened a new log
I20260812 07:03:22.488873 26564 ts_tablet_manager.cc:1397] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 07:03:22.489665 26564 raft_consensus.cc:348] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:22.489923 26564 raft_consensus.cc:374] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:22.490010 26564 raft_consensus.cc:729] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beb93a68b67948b285012d477c962c4c, State: Initialized, Role: FOLLOWER
I20260812 07:03:22.490268 26564 consensus_queue.cc:260] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:22.491345 26564 ts_tablet_manager.cc:1428] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 07:03:22.492259 26566 tablet_bootstrap.cc:492] T 335bb234196b46b7aa570e6669892348 P e2b70782d5814ce8a9beb0f8712762c9: No bootstrap required, opened a new log
I20260812 07:03:22.492443 26566 ts_tablet_manager.cc:1397] T 335bb234196b46b7aa570e6669892348 P e2b70782d5814ce8a9beb0f8712762c9: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20260812 07:03:22.493247 26566 raft_consensus.cc:348] T 335bb234196b46b7aa570e6669892348 P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:22.493495 26566 raft_consensus.cc:374] T 335bb234196b46b7aa570e6669892348 P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:22.493584 26566 raft_consensus.cc:729] T 335bb234196b46b7aa570e6669892348 P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b70782d5814ce8a9beb0f8712762c9, State: Initialized, Role: FOLLOWER
I20260812 07:03:22.493781 26566 consensus_queue.cc:260] T 335bb234196b46b7aa570e6669892348 P e2b70782d5814ce8a9beb0f8712762c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:22.494149 26566 ts_tablet_manager.cc:1428] T 335bb234196b46b7aa570e6669892348 P e2b70782d5814ce8a9beb0f8712762c9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 07:03:22.759336 26575 raft_consensus.cc:482] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:03:22.759631 26575 raft_consensus.cc:504] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:22.761185 25962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335bb234196b46b7aa570e6669892348" candidate_uuid: "beb93a68b67948b285012d477c962c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b70782d5814ce8a9beb0f8712762c9" is_pre_election: true
I20260812 07:03:22.761637 25962 raft_consensus.cc:2454] T 335bb234196b46b7aa570e6669892348 P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate beb93a68b67948b285012d477c962c4c in term 0.
I20260812 07:03:22.762185 26315 leader_election.cc:302] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: beb93a68b67948b285012d477c962c4c, e2b70782d5814ce8a9beb0f8712762c9; no voters:
I20260812 07:03:22.762482 26668 raft_consensus.cc:2781] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:03:22.762593 26668 raft_consensus.cc:482] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:03:22.762703 26668 raft_consensus.cc:3037] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:22.766053 26668 raft_consensus.cc:504] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:22.767243 25962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335bb234196b46b7aa570e6669892348" candidate_uuid: "beb93a68b67948b285012d477c962c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2b70782d5814ce8a9beb0f8712762c9"
I20260812 07:03:22.767436 25962 raft_consensus.cc:3037] T 335bb234196b46b7aa570e6669892348 P e2b70782d5814ce8a9beb0f8712762c9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:22.770663 25962 raft_consensus.cc:2454] T 335bb234196b46b7aa570e6669892348 P e2b70782d5814ce8a9beb0f8712762c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate beb93a68b67948b285012d477c962c4c in term 1.
I20260812 07:03:22.771509 26216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335bb234196b46b7aa570e6669892348" candidate_uuid: "beb93a68b67948b285012d477c962c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "162f4d9eeba348f9a9e103b57d8a85ae"
I20260812 07:03:22.771739 26216 raft_consensus.cc:3037] T 335bb234196b46b7aa570e6669892348 P 162f4d9eeba348f9a9e103b57d8a85ae [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:22.774660 26216 raft_consensus.cc:2454] T 335bb234196b46b7aa570e6669892348 P 162f4d9eeba348f9a9e103b57d8a85ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate beb93a68b67948b285012d477c962c4c in term 1.
I20260812 07:03:22.775559 26322 leader_election.cc:302] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 162f4d9eeba348f9a9e103b57d8a85ae, beb93a68b67948b285012d477c962c4c; no voters:
I20260812 07:03:22.775848 26669 raft_consensus.cc:2781] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:03:22.776046 26669 raft_consensus.cc:686] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c [term 1 LEADER]: Becoming Leader. State: Replica: beb93a68b67948b285012d477c962c4c, State: Running, Role: LEADER
I20260812 07:03:22.776350 26669 consensus_queue.cc:237] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } }
I20260812 07:03:22.777858 26668 leader_election.cc:288] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c [CANDIDATE]: Term 1 election: Requested vote from peers e2b70782d5814ce8a9beb0f8712762c9 (127.11.192.193:40691), 162f4d9eeba348f9a9e103b57d8a85ae (127.11.192.194:36263)
I20260812 07:03:22.778532 26216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335bb234196b46b7aa570e6669892348" candidate_uuid: "beb93a68b67948b285012d477c962c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" is_pre_election: true
I20260812 07:03:22.778787 26216 raft_consensus.cc:2362] T 335bb234196b46b7aa570e6669892348 P 162f4d9eeba348f9a9e103b57d8a85ae [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate beb93a68b67948b285012d477c962c4c in term 1. Re-sending same reply.
I20260812 07:03:22.779364 26575 leader_election.cc:288] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2b70782d5814ce8a9beb0f8712762c9 (127.11.192.193:40691), 162f4d9eeba348f9a9e103b57d8a85ae (127.11.192.194:36263)
I20260812 07:03:22.790112 25694 catalog_manager.cc:5495] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c reported cstate change: term changed from 0 to 1, leader changed from <none> to beb93a68b67948b285012d477c962c4c (127.11.192.195). New cstate: current_term: 1 leader_uuid: "beb93a68b67948b285012d477c962c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "beb93a68b67948b285012d477c962c4c" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 34345 } health_report { overall_health: HEALTHY } } }
I20260812 07:03:22.875492 26652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:22.870402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:43911 (local address 127.11.192.195:34345)
0812 07:03:22.870548 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:22.870552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:22.870915 (+ 363us) server_negotiation.cc:407] Connection header received
0812 07:03:22.871069 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:22.871073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:22.871168 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:22.871235 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:03:22.871642 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.872421 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.873768 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.874211 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.874297 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:22.874667 (+ 370us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:03:22.874789 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:03:22.874861 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:22.874992 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 07:03:22.875123 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":48}
I20260812 07:03:22.875526 26744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:22.870318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:34345 (local address 127.11.192.195:43911)
0812 07:03:22.870821 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:22.870837 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:22.870956 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:22.871260 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:22.871263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:22.871272 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:03:22.871529 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:22.871536 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.872540 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.872543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:22.873566 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:22.873578 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.874364 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.874369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:22.874499 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:22.874534 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:03:22.874879 (+ 345us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:03:22.874890 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 07:03:22.874955 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 07:03:22.875132 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":31,"thread_start_us":247,"threads_started":1}
I20260812 07:03:22.882606 26216 raft_consensus.cc:1261] T 335bb234196b46b7aa570e6669892348 P 162f4d9eeba348f9a9e103b57d8a85ae [term 1 FOLLOWER]: Refusing update from remote peer beb93a68b67948b285012d477c962c4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:03:22.882771 25962 raft_consensus.cc:1261] T 335bb234196b46b7aa570e6669892348 P e2b70782d5814ce8a9beb0f8712762c9 [term 1 FOLLOWER]: Refusing update from remote peer beb93a68b67948b285012d477c962c4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:03:22.883095 26575 consensus_queue.cc:1035] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "162f4d9eeba348f9a9e103b57d8a85ae" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:03:22.883395 26640 consensus_queue.cc:1035] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b70782d5814ce8a9beb0f8712762c9" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:03:22.926415 25693 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:03:22.926617 25693 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:03:22.929131 25693 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67b1650a4f4a4768b5412d496251a883: Sending DeleteTablet for 3 replicas of tablet 335bb234196b46b7aa570e6669892348
I20260812 07:03:22.929759 25932 tablet_service.cc:1514] Processing DeleteTablet for tablet 335bb234196b46b7aa570e6669892348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:03:22 UTC) from {username='oryx', principal='oryx/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:56944
I20260812 07:03:22.929864 26165 tablet_service.cc:1514] Processing DeleteTablet for tablet 335bb234196b46b7aa570e6669892348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:03:22 UTC) from {username='oryx', principal='oryx/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:45256
I20260812 07:03:22.929868 26411 tablet_service.cc:1514] Processing DeleteTablet for tablet 335bb234196b46b7aa570e6669892348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:03:22 UTC) from {username='oryx', principal='oryx/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:54354
I20260812 07:03:22.930042 26689 tablet_replica.cc:330] T 335bb234196b46b7aa570e6669892348 P e2b70782d5814ce8a9beb0f8712762c9: stopping tablet replica
I20260812 07:03:22.930114 26691 tablet_replica.cc:330] T 335bb234196b46b7aa570e6669892348 P 162f4d9eeba348f9a9e103b57d8a85ae: stopping tablet replica
I20260812 07:03:22.930150 26695 tablet_replica.cc:330] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c: stopping tablet replica
I20260812 07:03:22.930217 26689 raft_consensus.cc:2229] T 335bb234196b46b7aa570e6669892348 P e2b70782d5814ce8a9beb0f8712762c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:03:22.930267 26691 raft_consensus.cc:2229] T 335bb234196b46b7aa570e6669892348 P 162f4d9eeba348f9a9e103b57d8a85ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:03:22.930307 26695 raft_consensus.cc:2229] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:03:22.930410 26689 raft_consensus.cc:2258] T 335bb234196b46b7aa570e6669892348 P e2b70782d5814ce8a9beb0f8712762c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:03:22.930452 26691 raft_consensus.cc:2258] T 335bb234196b46b7aa570e6669892348 P 162f4d9eeba348f9a9e103b57d8a85ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:03:22.930610 26695 raft_consensus.cc:2258] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:03:22.930787 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 25822
I20260812 07:03:22.931529 26691 ts_tablet_manager.cc:1905] T 335bb234196b46b7aa570e6669892348 P 162f4d9eeba348f9a9e103b57d8a85ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:03:22.931731 26695 ts_tablet_manager.cc:1905] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:03:22.938365 26695 ts_tablet_manager.cc:1918] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:03:22.938508 26695 log.cc:1196] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/335bb234196b46b7aa570e6669892348
I20260812 07:03:22.938621 26691 ts_tablet_manager.cc:1918] T 335bb234196b46b7aa570e6669892348 P 162f4d9eeba348f9a9e103b57d8a85ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:03:22.938766 26691 log.cc:1196] T 335bb234196b46b7aa570e6669892348 P 162f4d9eeba348f9a9e103b57d8a85ae: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/335bb234196b46b7aa570e6669892348
I20260812 07:03:22.938946 26695 ts_tablet_manager.cc:1939] T 335bb234196b46b7aa570e6669892348 P beb93a68b67948b285012d477c962c4c: Deleting consensus metadata
I20260812 07:03:22.939164 26691 ts_tablet_manager.cc:1939] T 335bb234196b46b7aa570e6669892348 P 162f4d9eeba348f9a9e103b57d8a85ae: Deleting consensus metadata
I20260812 07:03:22.939602 26689 ts_tablet_manager.cc:1905] T 335bb234196b46b7aa570e6669892348 P e2b70782d5814ce8a9beb0f8712762c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:03:22.940160 25667 catalog_manager.cc:4841] TS 162f4d9eeba348f9a9e103b57d8a85ae (127.11.192.194:36263): tablet 335bb234196b46b7aa570e6669892348 (table test-table [id=6d222719e7de432ab783d7511e1a8adf]) successfully deleted
I20260812 07:03:22.940507 25665 catalog_manager.cc:4841] TS beb93a68b67948b285012d477c962c4c (127.11.192.195:34345): tablet 335bb234196b46b7aa570e6669892348 (table test-table [id=6d222719e7de432ab783d7511e1a8adf]) successfully deleted
I20260812 07:03:22.945945 26689 ts_tablet_manager.cc:1918] T 335bb234196b46b7aa570e6669892348 P e2b70782d5814ce8a9beb0f8712762c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:03:22.946094 26689 log.cc:1196] T 335bb234196b46b7aa570e6669892348 P e2b70782d5814ce8a9beb0f8712762c9: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518136653111-12035-0/minicluster-data/ts-0/wal/wals/335bb234196b46b7aa570e6669892348
I20260812 07:03:22.946502 26689 ts_tablet_manager.cc:1939] T 335bb234196b46b7aa570e6669892348 P e2b70782d5814ce8a9beb0f8712762c9: Deleting consensus metadata
I20260812 07:03:22.947505 25663 catalog_manager.cc:4841] TS e2b70782d5814ce8a9beb0f8712762c9 (127.11.192.193:40691): tablet 335bb234196b46b7aa570e6669892348 (table test-table [id=6d222719e7de432ab783d7511e1a8adf]) successfully deleted
Aug 12 07:03:22 dist-test-slave-lgr5 krb5kdc[25605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.192.193@KRBTEST.COM
I20260812 07:03:22.955478 26749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:22.932063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56950 (local address 127.11.192.193:40691)
0812 07:03:22.940742 (+ 8679us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:22.940746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:22.940761 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:03:22.940806 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:22.940809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:22.940868 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:22.940991 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:22.942594 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.943723 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.945460 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:22.945906 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:22.946001 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:22.950113 (+ 4112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:22.950128 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:22.950144 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:22.950176 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:22.952933 (+ 2757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:22.953549 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:22.953556 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:22.953558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:22.953692 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:22.954046 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:22.954050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:22.954052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:22.954386 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:22.954665 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:22.954947 (+ 282us) server_negotiation.cc:299] Negotiation successful
0812 07:03:22.955178 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8563,"thread_start_us":8445,"threads_started":1}
I20260812 07:03:22.956110 26050 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:23.078105 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 25822
I20260812 07:03:23.109395 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 26063
Aug 12 07:03:23 dist-test-slave-lgr5 krb5kdc[25605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.192.194@KRBTEST.COM
I20260812 07:03:23.123994 26752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:23.110114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45268 (local address 127.11.192.194:36263)
0812 07:03:23.110544 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:23.110548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:23.110564 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:03:23.110622 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:23.110625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:23.110689 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:23.110777 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:23.111991 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:23.112738 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:23.113949 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:23.114351 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:23.114448 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:23.118526 (+ 4078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:23.118542 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:23.118561 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:23.118600 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:23.121460 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:23.122133 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:23.122140 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:23.122142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:23.122278 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:23.122568 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:23.122571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:23.122573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:23.122963 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:23.123192 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:23.123435 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 07:03:23.123619 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":227,"threads_started":1}
I20260812 07:03:23.124686 26277 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:23.220214 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 26063
I20260812 07:03:23.251417 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 26289
Aug 12 07:03:23 dist-test-slave-lgr5 krb5kdc[25605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.192.195@KRBTEST.COM
I20260812 07:03:23.265538 26652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:23.252221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54390 (local address 127.11.192.195:34345)
0812 07:03:23.252400 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:23.252407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:23.252426 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:03:23.252526 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:23.252531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:23.252594 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:23.252679 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:23.253888 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:23.254564 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:23.255798 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:23.256267 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:23.256377 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:23.260292 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:23.260308 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:23.260325 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:23.260362 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:23.262927 (+ 2565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:23.263546 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:23.263554 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:23.263556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:23.263689 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:23.264038 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:23.264043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:23.264046 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:23.264459 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:23.264730 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:23.265025 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 07:03:23.265243 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:03:23.266144 26521 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:23.409814 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 26289
I20260812 07:03:23.443840 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 25634
I20260812 07:03:23.444968 25771 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:23.558070 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 25634
2026-08-12T07:03:23Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6269 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:03:23 dist-test-slave-lgr5 krb5kdc[26766](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:03:23 dist-test-slave-lgr5 krb5kdc[26766](info): set up 2 sockets
Aug 12 07:03:23 dist-test-slave-lgr5 krb5kdc[26766](info): commencing operation
krb5kdc: starting...
W20260812 07:03:24.804644 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:03:24 dist-test-slave-lgr5 krb5kdc[26766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:03:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:03:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:03:25.026883 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:41959
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:41959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:25.352283 26791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:25.352557 26791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:25.352698 26791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:25.359831 26791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:03:25.359944 26791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:25.360031 26791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:25.360129 26791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:03:25.360206 26791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:03:25.370292 26791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:41959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:41959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:25.371613 26791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:25.372906 26791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:25.383786 26800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:25.383859 26798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:25.383842 26797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:25.384927 26791 server_base.cc:1034] running on GCE node
I20260812 07:03:25.385654 26791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:25.386919 26791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:25.388144 26791 hybrid_clock.cc:648] HybridClock initialized: now 1786518205388093 us; error 32 us; skew 500 ppm
I20260812 07:03:25.388463 26791 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T07:03:25Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1887 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:03:25 dist-test-slave-lgr5 krb5kdc[26810](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:03:25 dist-test-slave-lgr5 krb5kdc[26810](info): set up 2 sockets
Aug 12 07:03:25 dist-test-slave-lgr5 krb5kdc[26810](info): commencing operation
krb5kdc: starting...
W20260812 07:03:26.683391 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:03:26 dist-test-slave-lgr5 krb5kdc[26810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:03:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:03:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:03:26.895555 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:39169
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:39169
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:27.212422 26834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:27.212731 26834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:27.212874 26834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:27.220062 26834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:03:27.220207 26834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:27.220285 26834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:27.220372 26834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:03:27.220436 26834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:03:27.230005 26834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:39169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:39169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:27.231249 26834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:27.232584 26834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:27.243657 26842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:27.243695 26843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:27.243732 26846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:27.244551 26834 server_base.cc:1034] running on GCE node
I20260812 07:03:27.245206 26834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:27.246438 26834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:27.247632 26834 hybrid_clock.cc:648] HybridClock initialized: now 1786518207247581 us; error 40 us; skew 500 ppm
I20260812 07:03:27.247942 26834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:27 dist-test-slave-lgr5 krb5kdc[26810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518207, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:27.250845 26834 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:03:27.252197 26834 webserver.cc:460] Webserver started at http://127.11.192.254:37491/ using document root <none> and password file <none>
I20260812 07:03:27.252815 26834 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:27.252910 26834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:27.253157 26834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:27.255939 26834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "705fff75b37b4bffb934d3e3701fc031"
format_stamp: "Formatted at 2026-08-12 07:03:27 on dist-test-slave-lgr5"
server_key: "19aa5c9709f7dc6b0a8bf6c601266e1e"
server_key_iv: "291cbc855e955fce1b7f578064acdb66"
server_key_version: "encryptionkey@0"
I20260812 07:03:27.256688 26834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "705fff75b37b4bffb934d3e3701fc031"
format_stamp: "Formatted at 2026-08-12 07:03:27 on dist-test-slave-lgr5"
server_key: "19aa5c9709f7dc6b0a8bf6c601266e1e"
server_key_iv: "291cbc855e955fce1b7f578064acdb66"
server_key_version: "encryptionkey@0"
I20260812 07:03:27.261402 26834 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:03:27.264545 26858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:27.265713 26834 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260812 07:03:27.265857 26834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "705fff75b37b4bffb934d3e3701fc031"
format_stamp: "Formatted at 2026-08-12 07:03:27 on dist-test-slave-lgr5"
server_key: "19aa5c9709f7dc6b0a8bf6c601266e1e"
server_key_iv: "291cbc855e955fce1b7f578064acdb66"
server_key_version: "encryptionkey@0"
I20260812 07:03:27.266021 26834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:27.293406 26834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:27.297077 26834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:27.297348 26834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:27.313788 26834 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:39169
I20260812 07:03:27.313807 26945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:39169 every 8 connection(s)
I20260812 07:03:27.315064 26834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:03:27.318580 26947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:27.325340 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 26834
I20260812 07:03:27.325577 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:03:27.326342 12035 external_mini_cluster.cc:1428] Setting key 338076bd23ddf64120a1dcec2b0c4434
I20260812 07:03:27.329210 26947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031: Bootstrap starting.
I20260812 07:03:27.332376 26947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:27.333578 26947 log.cc:824] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031: Log is configured to *not* fsync() on all Append() calls
I20260812 07:03:27.336112 26947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031: No bootstrap required, opened a new log
Aug 12 07:03:27 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:03:27.344595 26947 raft_consensus.cc:348] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705fff75b37b4bffb934d3e3701fc031" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39169 } }
I20260812 07:03:27.344928 26947 raft_consensus.cc:374] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:27.345045 26947 raft_consensus.cc:729] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 705fff75b37b4bffb934d3e3701fc031, State: Initialized, Role: FOLLOWER
I20260812 07:03:27.345676 26947 consensus_queue.cc:260] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705fff75b37b4bffb934d3e3701fc031" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39169 } }
I20260812 07:03:27.345965 26947 raft_consensus.cc:388] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:03:27.346108 26947 raft_consensus.cc:482] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:03:27.346297 26947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:27.348629 26956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:27.328304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54558 (local address 127.11.192.254:39169)
0812 07:03:27.328993 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:27.329004 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:27.329043 (+ 39us) server_negotiation.cc:407] Connection header received
0812 07:03:27.330742 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:27.330765 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:27.331106 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:27.331399 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:27.333251 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:27.334557 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:27.335788 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:27.336535 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:27.336703 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:27.340413 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:27.340427 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:27.340449 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:27.340490 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:27.344280 (+ 3790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:27.344927 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:27.344935 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:27.344943 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:27.345164 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:27.345504 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:27.345509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:27.345513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:27.346217 (+ 704us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:27.346511 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:27.346906 (+ 395us) server_negotiation.cc:299] Negotiation successful
0812 07:03:27.347335 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":309,"threads_started":1}
I20260812 07:03:27.349352 26947 raft_consensus.cc:504] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705fff75b37b4bffb934d3e3701fc031" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39169 } }
I20260812 07:03:27.349797 26947 leader_election.cc:302] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 705fff75b37b4bffb934d3e3701fc031; no voters:
I20260812 07:03:27.350378 26947 leader_election.cc:288] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:03:27.350497 26959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:03:27.352332 26959 raft_consensus.cc:686] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [term 1 LEADER]: Becoming Leader. State: Replica: 705fff75b37b4bffb934d3e3701fc031, State: Running, Role: LEADER
I20260812 07:03:27.352509 26947 sys_catalog.cc:564] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:03:27.352834 26959 consensus_queue.cc:237] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705fff75b37b4bffb934d3e3701fc031" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39169 } }
I20260812 07:03:27.355679 26960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "705fff75b37b4bffb934d3e3701fc031" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705fff75b37b4bffb934d3e3701fc031" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39169 } } }
I20260812 07:03:27.356184 26960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:27.355654 26961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 705fff75b37b4bffb934d3e3701fc031. Latest consensus state: current_term: 1 leader_uuid: "705fff75b37b4bffb934d3e3701fc031" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705fff75b37b4bffb934d3e3701fc031" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39169 } } }
I20260812 07:03:27.356768 26961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [sys.catalog]: This master's current role is: LEADER
W20260812 07:03:27.361325 26989 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:03:27.361418 26989 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:03:27.361517 26983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:03:27.363619 26983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:03:27.371723 26983 catalog_manager.cc:1347] Generated new cluster ID: 44a39558bcd944289048bb83db3a7dad
I20260812 07:03:27.371824 26983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:03:27.380214 26983 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:03:27.381711 26983 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:03:27.388798 26983 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031: Generated new TSK 0
I20260812 07:03:27.389452 26983 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
I20260812 07:03:27.453590 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:39169
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:27.772905 27000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:27.773149 27000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:27.773244 27000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:27.780058 27000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:27.780166 27000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:27.780277 27000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:03:27.789202 27000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:39169
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:27.790246 27000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:27.791406 27000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:27.803435 27018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:27.803507 27014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:27.803541 27010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:27.804399 27000 server_base.cc:1034] running on GCE node
I20260812 07:03:27.804919 27000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:27.805759 27000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:27.806955 27000 hybrid_clock.cc:648] HybridClock initialized: now 1786518207806904 us; error 37 us; skew 500 ppm
I20260812 07:03:27.807248 27000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:27 dist-test-slave-lgr5 krb5kdc[26810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518207, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:27.810148 27000 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:03:27.811431 27000 webserver.cc:460] Webserver started at http://127.11.192.193:45129/ using document root <none> and password file <none>
I20260812 07:03:27.812047 27000 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:27.812180 27000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:27.812462 27000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:27.815124 27000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "91e567e5990c424a87dc8e49705c343a"
format_stamp: "Formatted at 2026-08-12 07:03:27 on dist-test-slave-lgr5"
server_key: "f05ad242551e688dfe4edab4884655f1"
server_key_iv: "54df4423ec029524e7ec406e010ea307"
server_key_version: "encryptionkey@0"
I20260812 07:03:27.815860 27000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "91e567e5990c424a87dc8e49705c343a"
format_stamp: "Formatted at 2026-08-12 07:03:27 on dist-test-slave-lgr5"
server_key: "f05ad242551e688dfe4edab4884655f1"
server_key_iv: "54df4423ec029524e7ec406e010ea307"
server_key_version: "encryptionkey@0"
I20260812 07:03:27.820611 27000 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:03:27.823714 27029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:27.824872 27000 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:03:27.825027 27000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "91e567e5990c424a87dc8e49705c343a"
format_stamp: "Formatted at 2026-08-12 07:03:27 on dist-test-slave-lgr5"
server_key: "f05ad242551e688dfe4edab4884655f1"
server_key_iv: "54df4423ec029524e7ec406e010ea307"
server_key_version: "encryptionkey@0"
I20260812 07:03:27.825203 27000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:27.839148 27000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:27.842645 27000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:27.842885 27000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:27.843709 27000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:27.844937 27000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:27.845029 27000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:27.845144 27000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:27.845201 27000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:27.874332 27000 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:42407
I20260812 07:03:27.874379 27236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:42407 every 8 connection(s)
I20260812 07:03:27.875581 27000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:03:27.877858 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 27000
I20260812 07:03:27.878096 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:03:27.878731 12035 external_mini_cluster.cc:1428] Setting key da70f8687f3442a7d464f09ea26c7fdb
Aug 12 07:03:27 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518207, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:03:27.895462 27244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:27.878037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:54511 (local address 127.11.192.254:39169)
0812 07:03:27.880424 (+ 2387us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:27.880428 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:27.880446 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:03:27.881829 (+ 1383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:27.881834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:27.881906 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:27.882023 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:27.883397 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:27.884383 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:27.885853 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:27.886378 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:27.886495 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:27.889827 (+ 3332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:27.889833 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:27.889837 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:27.889873 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:27.892526 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:27.893297 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:27.893300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:27.893302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:27.893413 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:27.893834 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:27.893837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:27.893839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:27.894097 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:27.894205 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:27.895021 (+ 816us) server_negotiation.cc:299] Negotiation successful
0812 07:03:27.895213 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2289,"thread_start_us":2179,"threads_started":1}
I20260812 07:03:27.896790 27245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:27.878614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.193:54511)
0812 07:03:27.879526 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:27.879575 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:27.881571 (+ 1996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:27.882185 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:27.882193 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:27.882686 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:27.883229 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:27.883246 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:27.884513 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:27.884517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:27.885712 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:27.885725 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:27.886552 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:27.886556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:27.886747 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:27.887351 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:27.887373 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:27.889610 (+ 2237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:27.892659 (+ 3049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:27.892668 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:27.892681 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:27.893201 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:27.893508 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:27.893512 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:27.893515 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:27.893748 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:27.894215 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:27.894223 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:27.894622 (+ 399us) client_negotiation.cc:769] Sending connection context
0812 07:03:27.894938 (+ 316us) client_negotiation.cc:240] Negotiation successful
0812 07:03:27.895448 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":361,"threads_started":1}
I20260812 07:03:27.898321 27240 heartbeater.cc:347] Connected to a master server at 127.11.192.254:39169
I20260812 07:03:27.898605 27240 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:27.899293 27240 heartbeater.cc:511] Master 127.11.192.254:39169 requested a full tablet report, sending...
I20260812 07:03:27.901234 26893 ts_manager.cc:194] Registered new tserver with Master: 91e567e5990c424a87dc8e49705c343a (127.11.192.193:42407)
I20260812 07:03:27.903118 26893 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:54511
WARNING: no policy specified for kudu/127.11.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
I20260812 07:03:27.934087 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:39169
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:28.253515 27250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:28.253784 27250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:28.253887 27250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:28.260934 27250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:28.261041 27250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:28.261159 27250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:03:28.270506 27250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:39169
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:28.271736 27250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:28.273161 27250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:28.285606 27259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:28.285679 27258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:28.285642 27263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:28.286311 27250 server_base.cc:1034] running on GCE node
I20260812 07:03:28.286891 27250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:28.287737 27250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:28.288928 27250 hybrid_clock.cc:648] HybridClock initialized: now 1786518208288868 us; error 50 us; skew 500 ppm
I20260812 07:03:28.289233 27250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:28 dist-test-slave-lgr5 krb5kdc[26810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518208, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:28.292147 27250 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:03:28.293499 27250 webserver.cc:460] Webserver started at http://127.11.192.194:45335/ using document root <none> and password file <none>
I20260812 07:03:28.294134 27250 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:28.294250 27250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:28.294534 27250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:28.297396 27250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "d467cb0d29164b2cbd1a53643f54cc79"
format_stamp: "Formatted at 2026-08-12 07:03:28 on dist-test-slave-lgr5"
server_key: "388613daf071b499b710fe8e75aca5af"
server_key_iv: "852c928ad04203f858811f3b9977fa0d"
server_key_version: "encryptionkey@0"
I20260812 07:03:28.298193 27250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "d467cb0d29164b2cbd1a53643f54cc79"
format_stamp: "Formatted at 2026-08-12 07:03:28 on dist-test-slave-lgr5"
server_key: "388613daf071b499b710fe8e75aca5af"
server_key_iv: "852c928ad04203f858811f3b9977fa0d"
server_key_version: "encryptionkey@0"
I20260812 07:03:28.303016 27250 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:03:28.306252 27274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:28.307416 27250 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:03:28.307569 27250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "d467cb0d29164b2cbd1a53643f54cc79"
format_stamp: "Formatted at 2026-08-12 07:03:28 on dist-test-slave-lgr5"
server_key: "388613daf071b499b710fe8e75aca5af"
server_key_iv: "852c928ad04203f858811f3b9977fa0d"
server_key_version: "encryptionkey@0"
I20260812 07:03:28.307741 27250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:28.323232 27250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:28.327275 27250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:28.327533 27250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:28.328485 27250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:28.329830 27250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:28.329919 27250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:28.330037 27250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:28.330090 27250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:28.359740 27250 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:38645
I20260812 07:03:28.359782 27479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:38645 every 8 connection(s)
I20260812 07:03:28.360975 27250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:03:28.363603 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 27250
I20260812 07:03:28.363778 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:03:28.364318 12035 external_mini_cluster.cc:1428] Setting key 12ac39f0da5b9eb39d3ad4a45f868f85
Aug 12 07:03:28 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518208, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:03:28.381039 27244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:28.363403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:54057 (local address 127.11.192.254:39169)
0812 07:03:28.363630 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:28.363637 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:28.364914 (+ 1277us) server_negotiation.cc:407] Connection header received
0812 07:03:28.367094 (+ 2180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:28.367100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:28.367187 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:28.367282 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:28.368742 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:28.369715 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:28.371323 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:28.371858 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:28.371978 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:28.375269 (+ 3291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:28.375275 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:28.375279 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:28.375315 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:28.378020 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:28.378859 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:28.378864 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:28.378867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:28.379021 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:28.379390 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:28.379395 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:28.379399 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:28.379728 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:28.379868 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:28.380491 (+ 623us) server_negotiation.cc:299] Negotiation successful
0812 07:03:28.380717 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 07:03:28.381693 27483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:28.363842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.194:54057)
0812 07:03:28.364720 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:28.364775 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:28.366829 (+ 2054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:28.367471 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:28.367478 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:28.367974 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:28.368578 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:28.368597 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:28.369850 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:28.369854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:28.371171 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:28.371185 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:28.372049 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:28.372054 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:28.372291 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:28.372896 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:28.372917 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:28.375060 (+ 2143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:28.378150 (+ 3090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:28.378159 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:28.378173 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:28.378736 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:28.379107 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:28.379110 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:28.379112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:28.379285 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:28.379828 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:28.379834 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:28.380133 (+ 299us) client_negotiation.cc:769] Sending connection context
0812 07:03:28.380342 (+ 209us) client_negotiation.cc:240] Negotiation successful
0812 07:03:28.380691 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":316,"threads_started":1}
I20260812 07:03:28.382997 27480 heartbeater.cc:347] Connected to a master server at 127.11.192.254:39169
I20260812 07:03:28.383350 27480 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:28.384187 27480 heartbeater.cc:511] Master 127.11.192.254:39169 requested a full tablet report, sending...
I20260812 07:03:28.385727 26893 ts_manager.cc:194] Registered new tserver with Master: d467cb0d29164b2cbd1a53643f54cc79 (127.11.192.194:38645)
I20260812 07:03:28.386495 26893 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:54057
WARNING: no policy specified for kudu/127.11.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
I20260812 07:03:28.419844 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:39169
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:28.732856 27493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:28.733104 27493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:28.733202 27493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:28.739981 27493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:28.740091 27493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:28.740203 27493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:03:28.749253 27493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:39169
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:28.750319 27493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:28.751471 27493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:28.763628 27501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:28.763651 27502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:28.763686 27504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:28.764676 27493 server_base.cc:1034] running on GCE node
I20260812 07:03:28.765199 27493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:28.766016 27493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:28.767203 27493 hybrid_clock.cc:648] HybridClock initialized: now 1786518208767164 us; error 27 us; skew 500 ppm
I20260812 07:03:28.767489 27493 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:28 dist-test-slave-lgr5 krb5kdc[26810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518208, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:28.770393 27493 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:03:28.771713 27493 webserver.cc:460] Webserver started at http://127.11.192.195:32929/ using document root <none> and password file <none>
I20260812 07:03:28.772424 27493 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:28.772539 27493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:28.772814 27493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:28.775465 27493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "0670a15557d14b298f7142dc9bb913d2"
format_stamp: "Formatted at 2026-08-12 07:03:28 on dist-test-slave-lgr5"
server_key: "7163eaa4dcfc91cd115aa6b8ac670e7f"
server_key_iv: "5c27bc890f65758d4f77add1984d7259"
server_key_version: "encryptionkey@0"
I20260812 07:03:28.776240 27493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "0670a15557d14b298f7142dc9bb913d2"
format_stamp: "Formatted at 2026-08-12 07:03:28 on dist-test-slave-lgr5"
server_key: "7163eaa4dcfc91cd115aa6b8ac670e7f"
server_key_iv: "5c27bc890f65758d4f77add1984d7259"
server_key_version: "encryptionkey@0"
I20260812 07:03:28.781023 27493 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:03:28.784097 27517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:28.785326 27493 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 07:03:28.785472 27493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "0670a15557d14b298f7142dc9bb913d2"
format_stamp: "Formatted at 2026-08-12 07:03:28 on dist-test-slave-lgr5"
server_key: "7163eaa4dcfc91cd115aa6b8ac670e7f"
server_key_iv: "5c27bc890f65758d4f77add1984d7259"
server_key_version: "encryptionkey@0"
I20260812 07:03:28.785638 27493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:28.806380 27493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:28.809896 27493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:28.810144 27493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:28.811058 27493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:28.812405 27493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:28.812485 27493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:28.812604 27493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:28.812657 27493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:28.842195 27493 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:42243
I20260812 07:03:28.842253 27714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:42243 every 8 connection(s)
I20260812 07:03:28.843417 27493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:03:28.848156 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 27493
I20260812 07:03:28.848352 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:03:28.848896 12035 external_mini_cluster.cc:1428] Setting key 5b49c08ef6d6bbe73b708c92864d2455
Aug 12 07:03:28 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518208, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:03:28.862510 27244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:28.845821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:43467 (local address 127.11.192.254:39169)
0812 07:03:28.845995 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:28.845999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:28.847110 (+ 1111us) server_negotiation.cc:407] Connection header received
0812 07:03:28.849393 (+ 2283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:28.849398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:28.849488 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:28.849600 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:28.851366 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:28.852247 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:28.853757 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:28.854178 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:28.854267 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:28.857401 (+ 3134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:28.857406 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:28.857409 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:28.857440 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:28.860004 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:28.860645 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:28.860648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:28.860650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:28.860759 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:28.861090 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:28.861093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:28.861095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:28.861341 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:28.861443 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:28.862004 (+ 561us) server_negotiation.cc:299] Negotiation successful
0812 07:03:28.862155 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:03:28.863480 27717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:28.846258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.195:43467)
0812 07:03:28.846943 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:28.846988 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:28.849099 (+ 2111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:28.849828 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:28.849838 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:28.850550 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:28.851208 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:28.851226 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:28.852400 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:28.852404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:28.853643 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:28.853659 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:28.854352 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:28.854357 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:28.854545 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:28.855167 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:28.855194 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:28.857235 (+ 2041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:28.860147 (+ 2912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:28.860154 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:28.860165 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:28.860562 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:28.860833 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:28.860836 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:28.860838 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:28.861011 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:28.861424 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:28.861429 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:28.861710 (+ 281us) client_negotiation.cc:769] Sending connection context
0812 07:03:28.861929 (+ 219us) client_negotiation.cc:240] Negotiation successful
0812 07:03:28.862415 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":261,"threads_started":1}
I20260812 07:03:28.864629 27715 heartbeater.cc:347] Connected to a master server at 127.11.192.254:39169
I20260812 07:03:28.864898 27715 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:28.865549 27715 heartbeater.cc:511] Master 127.11.192.254:39169 requested a full tablet report, sending...
I20260812 07:03:28.866825 26893 ts_manager.cc:194] Registered new tserver with Master: 0670a15557d14b298f7142dc9bb913d2 (127.11.192.195:42243)
I20260812 07:03:28.867569 26893 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:43467
I20260812 07:03:28.875890 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.192.193@KRBTEST.COM" created.
I20260812 07:03:28.906548 27240 heartbeater.cc:503] Master 127.11.192.254:39169 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.193.keytab.
Entry for principal oryx/127.11.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.193.keytab.
WARNING: no policy specified for oryx/127.11.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.194.keytab.
Entry for principal oryx/127.11.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.194.keytab.
WARNING: no policy specified for oryx/127.11.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.195.keytab.
Entry for principal oryx/127.11.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.195.keytab.
I20260812 07:03:29.006630 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 27000
Aug 12 07:03:29 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.193@KRBTEST.COM
I20260812 07:03:29.021504 27735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:29.007657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44944 (local address 127.11.192.193:42407)
0812 07:03:29.008276 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:29.008284 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:29.008312 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:03:29.008377 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:29.008384 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:29.008610 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:29.008745 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:29.010012 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:29.011096 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:29.012342 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:29.012938 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:29.013036 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:29.016694 (+ 3658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:29.016706 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:29.016720 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:29.016756 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:29.019223 (+ 2467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:29.019832 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:29.019838 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:29.019840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:29.019958 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:29.020272 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:29.020276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:29.020278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:29.020614 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:29.020841 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:29.021022 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 07:03:29.021211 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":289,"threads_started":1}
I20260812 07:03:29.022784 27230 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:29.103452 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 27000
I20260812 07:03:29.122942 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 27250
Aug 12 07:03:29 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.194@KRBTEST.COM
I20260812 07:03:29.137475 27740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:29.123786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51052 (local address 127.11.192.194:38645)
0812 07:03:29.124291 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:29.124297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:29.124322 (+ 25us) server_negotiation.cc:407] Connection header received
0812 07:03:29.124384 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:29.124389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:29.124597 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:29.124718 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:29.125864 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:29.126843 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:29.128006 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:29.128603 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:29.128704 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:29.132521 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:29.132533 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:29.132546 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:29.132578 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:29.135063 (+ 2485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:29.135645 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:29.135651 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:29.135654 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:29.135764 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:29.136025 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:29.136027 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:29.136029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:29.136466 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:29.136698 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:29.136934 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 07:03:29.137129 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":233,"threads_started":1}
I20260812 07:03:29.138653 27469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:29.219954 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 27250
I20260812 07:03:29.242934 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 27493
Aug 12 07:03:29 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.195@KRBTEST.COM
I20260812 07:03:29.258783 27743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:29.243830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34572 (local address 127.11.192.195:42243)
0812 07:03:29.244471 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:29.244478 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:29.244505 (+ 27us) server_negotiation.cc:407] Connection header received
0812 07:03:29.244567 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:29.244574 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:29.244776 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:29.244930 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:29.246267 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:29.247304 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:29.248711 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:29.249284 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:29.249409 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:29.253671 (+ 4262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:29.253683 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:29.253698 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:29.253743 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:29.256624 (+ 2881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:29.257186 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:29.257192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:29.257194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:29.257316 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:29.257582 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:29.257585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:29.257587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:29.257897 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:29.258130 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:29.258310 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 07:03:29.258508 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":324,"threads_started":1}
I20260812 07:03:29.259994 27710 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:29.340127 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 27493
I20260812 07:03:29.364301 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 26834
I20260812 07:03:29.365437 26938 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:29.467444 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 26834
I20260812 07:03:29.490422 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:39169
--webserver_interface=127.11.192.254
--webserver_port=37491
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:39169
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:29.805155 27748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:29.805408 27748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:29.805502 27748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:29.812278 27748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:03:29.812371 27748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:29.812422 27748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:29.812477 27748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:03:29.812525 27748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:03:29.821673 27748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:39169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:39169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=37491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:29.822736 27748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:29.823897 27748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:29.834651 27761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:29.834700 27756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:29.834722 27757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:29.835502 27748 server_base.cc:1034] running on GCE node
I20260812 07:03:29.836186 27748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:29.837392 27748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:29.838598 27748 hybrid_clock.cc:648] HybridClock initialized: now 1786518209838543 us; error 39 us; skew 500 ppm
I20260812 07:03:29.838894 27748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:29 dist-test-slave-lgr5 krb5kdc[26810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518209, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:29.841727 27748 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:03:29.843107 27748 webserver.cc:460] Webserver started at http://127.11.192.254:37491/ using document root <none> and password file <none>
I20260812 07:03:29.843727 27748 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:29.843856 27748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:29.848924 27748 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:03:29.851557 27772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:29.852790 27748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:03:29.852932 27748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "705fff75b37b4bffb934d3e3701fc031"
format_stamp: "Formatted at 2026-08-12 07:03:27 on dist-test-slave-lgr5"
server_key: "19aa5c9709f7dc6b0a8bf6c601266e1e"
server_key_iv: "291cbc855e955fce1b7f578064acdb66"
server_key_version: "encryptionkey@0"
I20260812 07:03:29.853824 27748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:29.885628 27748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:29.889252 27748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:29.889467 27748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:29.906242 27748 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:39169
I20260812 07:03:29.906261 27875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:39169 every 8 connection(s)
I20260812 07:03:29.907622 27748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:03:29.912250 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 27748
I20260812 07:03:29.913003 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:42407
--local_ip_for_outbound_sockets=127.11.192.193
--tserver_master_addrs=127.11.192.254:39169
--webserver_port=45129
--webserver_interface=127.11.192.193
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.192.193
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:03:29.914222 27876 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:03:29.917330 27876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031: Bootstrap starting.
I20260812 07:03:29.935348 27876 log.cc:824] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031: Log is configured to *not* fsync() on all Append() calls
I20260812 07:03:29.943574 27876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:03:29.944044 27876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031: Bootstrap complete.
I20260812 07:03:29.952554 27876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705fff75b37b4bffb934d3e3701fc031" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39169 } }
I20260812 07:03:29.953343 27876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 705fff75b37b4bffb934d3e3701fc031, State: Initialized, Role: FOLLOWER
I20260812 07:03:29.953881 27876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705fff75b37b4bffb934d3e3701fc031" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39169 } }
I20260812 07:03:29.954109 27876 raft_consensus.cc:388] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:03:29.954211 27876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:03:29.954368 27876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [term 1 FOLLOWER]: Advancing to term 2
I20260812 07:03:29.958118 27876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705fff75b37b4bffb934d3e3701fc031" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39169 } }
I20260812 07:03:29.958559 27876 leader_election.cc:302] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 705fff75b37b4bffb934d3e3701fc031; no voters:
I20260812 07:03:29.959085 27876 leader_election.cc:288] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 07:03:29.959267 27882 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 07:03:29.960275 27882 raft_consensus.cc:686] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [term 2 LEADER]: Becoming Leader. State: Replica: 705fff75b37b4bffb934d3e3701fc031, State: Running, Role: LEADER
I20260812 07:03:29.960822 27882 consensus_queue.cc:237] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705fff75b37b4bffb934d3e3701fc031" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39169 } }
I20260812 07:03:29.961072 27876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:03:29.963210 27884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 705fff75b37b4bffb934d3e3701fc031. Latest consensus state: current_term: 2 leader_uuid: "705fff75b37b4bffb934d3e3701fc031" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705fff75b37b4bffb934d3e3701fc031" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39169 } } }
I20260812 07:03:29.963641 27884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:29.965343 27883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "705fff75b37b4bffb934d3e3701fc031" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705fff75b37b4bffb934d3e3701fc031" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39169 } } }
I20260812 07:03:29.966946 27883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:29.977315 27894 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:03:29.996613 27894 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:03:29.998019 27894 catalog_manager.cc:1259] Loaded cluster ID: 44a39558bcd944289048bb83db3a7dad
I20260812 07:03:29.998116 27894 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:03:30.001929 27894 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:03:30.003393 27894 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 705fff75b37b4bffb934d3e3701fc031: Loaded TSK: 0
I20260812 07:03:30.004858 27894 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 07:03:30.254676 27879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:30.254956 27879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:30.255105 27879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:30.262285 27879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:30.262389 27879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:30.262534 27879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:03:30.271697 27879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.193.keytab
--principal=oryx/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:42407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=45129
--tserver_master_addrs=127.11.192.254:39169
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:30.272897 27879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:30.274111 27879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:30.287122 27925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:30.287114 27926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:30.287727 27928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:30.289160 27879 server_base.cc:1034] running on GCE node
I20260812 07:03:30.289784 27879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:30.290681 27879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:30.291884 27879 hybrid_clock.cc:648] HybridClock initialized: now 1786518210291830 us; error 52 us; skew 500 ppm
I20260812 07:03:30.292222 27879 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:30 dist-test-slave-lgr5 krb5kdc[26810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518210, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:30.295390 27879 init.cc:377] Logged in from keytab as oryx/127.11.192.193@KRBTEST.COM (short username oryx)
I20260812 07:03:30.296973 27879 webserver.cc:460] Webserver started at http://127.11.192.193:45129/ using document root <none> and password file <none>
I20260812 07:03:30.297644 27879 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:30.297775 27879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:30.303148 27879 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:03:30.305996 27943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:30.307394 27879 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:03:30.307547 27879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "91e567e5990c424a87dc8e49705c343a"
format_stamp: "Formatted at 2026-08-12 07:03:27 on dist-test-slave-lgr5"
server_key: "f05ad242551e688dfe4edab4884655f1"
server_key_iv: "54df4423ec029524e7ec406e010ea307"
server_key_version: "encryptionkey@0"
I20260812 07:03:30.308497 27879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:30.329900 27879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:30.333556 27879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:30.333788 27879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:30.334678 27879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:30.335952 27879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:30.336058 27879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:30.336187 27879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:30.336257 27879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:30.365816 27879 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:42407
I20260812 07:03:30.365885 28140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:42407 every 8 connection(s)
I20260812 07:03:30.367143 27879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:03:30.368319 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 27879
I20260812 07:03:30.369060 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:38645
--local_ip_for_outbound_sockets=127.11.192.194
--tserver_master_addrs=127.11.192.254:39169
--webserver_port=45335
--webserver_interface=127.11.192.194
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.192.194
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:03:30 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:30 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:30.386462 28151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:30.370416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.193:56657)
0812 07:03:30.372553 (+ 2137us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:30.372588 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:30.374207 (+ 1619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:30.375627 (+ 1420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:30.375637 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:30.376300 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:30.376811 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:30.376829 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.378341 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.378344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:30.379530 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:30.379543 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.380611 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.380619 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:30.380863 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:30.381534 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:30.381568 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:30.385311 (+ 3743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1893,"thread_start_us":1715,"threads_started":1}
W20260812 07:03:30.386696 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:30.370194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:56657 (local address 127.11.192.254:39169)
0812 07:03:30.370837 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:30.370848 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:30.372720 (+ 1872us) server_negotiation.cc:407] Connection header received
0812 07:03:30.374770 (+ 2050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:30.374788 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:30.375120 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:30.375434 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:30.377003 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.378190 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.379663 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.380377 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.380562 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:30.385663 (+ 5101us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.193:56657: BlockingRecv error: recv got EOF from 127.11.192.193:56657 (error 108)
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":247,"threads_started":1}
W20260812 07:03:30.387306 28142 heartbeater.cc:644] Failed to heartbeat to 127.11.192.254:39169 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.192.254:39169: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Aug 12 07:03:30 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:30 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:30.394762 28151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:30.387853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.193:42155)
0812 07:03:30.388022 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:30.388040 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:30.388154 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:30.388431 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:30.388435 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:30.388740 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:30.388869 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:30.388876 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.389815 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.389819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:30.390673 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:30.390684 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.391330 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.391336 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:30.391498 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:30.391987 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:30.392007 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:30.394502 (+ 2495us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 07:03:30.394896 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:30.387924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:42155 (local address 127.11.192.254:39169)
0812 07:03:30.388096 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:30.388102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:30.388118 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:03:30.388255 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:30.388260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:30.388319 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:30.388393 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:30.389003 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.389683 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.390800 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.391198 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.391302 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:30.394698 (+ 3396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.193:42155: BlockingRecv error: recv got EOF from 127.11.192.193:42155 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 07:03:30 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:30 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:30.402354 28151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:30.395585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.193:37635)
0812 07:03:30.395721 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:30.395738 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:30.395810 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:30.396222 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:30.396227 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:30.396516 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:30.396648 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:30.396656 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.397596 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.397601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:30.398453 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:30.398464 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.399113 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.399119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:30.399268 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:30.399749 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:30.399766 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:30.402099 (+ 2333us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 07:03:30.402524 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:30.395790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:37635 (local address 127.11.192.254:39169)
0812 07:03:30.395948 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:30.395952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:30.395965 (+ 13us) server_negotiation.cc:407] Connection header received
0812 07:03:30.396013 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:30.396016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:30.396100 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:30.396185 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:30.396775 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.397458 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.398591 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.398976 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.399087 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:30.402296 (+ 3209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.193:37635: BlockingRecv error: recv got EOF from 127.11.192.193:37635 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260812 07:03:30.402878 28142 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 07:03:30.690448 28155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:30.690685 28155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:30.690780 28155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:30.697642 28155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:30.697732 28155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:30.697844 28155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:03:30.706871 28155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.194.keytab
--principal=oryx/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:38645
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=45335
--tserver_master_addrs=127.11.192.254:39169
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:30.707917 28155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:30.709123 28155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:30.721177 28160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:30.721256 28167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:30.721215 28162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:30.721936 28155 server_base.cc:1034] running on GCE node
I20260812 07:03:30.722499 28155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:30.723351 28155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:30.724543 28155 hybrid_clock.cc:648] HybridClock initialized: now 1786518210724492 us; error 33 us; skew 500 ppm
I20260812 07:03:30.724881 28155 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:30 dist-test-slave-lgr5 krb5kdc[26810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518210, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:30.727684 28155 init.cc:377] Logged in from keytab as oryx/127.11.192.194@KRBTEST.COM (short username oryx)
I20260812 07:03:30.728982 28155 webserver.cc:460] Webserver started at http://127.11.192.194:45335/ using document root <none> and password file <none>
I20260812 07:03:30.729580 28155 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:30.729704 28155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:30.734678 28155 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260812 07:03:30.737160 28185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:30.738329 28155 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260812 07:03:30.738472 28155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "d467cb0d29164b2cbd1a53643f54cc79"
format_stamp: "Formatted at 2026-08-12 07:03:28 on dist-test-slave-lgr5"
server_key: "388613daf071b499b710fe8e75aca5af"
server_key_iv: "852c928ad04203f858811f3b9977fa0d"
server_key_version: "encryptionkey@0"
I20260812 07:03:30.739346 28155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:30.757254 28155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:30.761013 28155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:30.761272 28155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:30.762215 28155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:30.763540 28155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:30.763648 28155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:30.763768 28155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:30.763850 28155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:30.795188 28155 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:38645
I20260812 07:03:30.795248 28397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:38645 every 8 connection(s)
I20260812 07:03:30.796669 28155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:03:30.805424 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 28155
I20260812 07:03:30.805943 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:42243
--local_ip_for_outbound_sockets=127.11.192.195
--tserver_master_addrs=127.11.192.254:39169
--webserver_port=32929
--webserver_interface=127.11.192.195
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.192.195
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:03:30 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:30 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 07:03:30.812131 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:30.798988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:36181 (local address 127.11.192.254:39169)
0812 07:03:30.799187 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:30.799193 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:30.800369 (+ 1176us) server_negotiation.cc:407] Connection header received
0812 07:03:30.802173 (+ 1804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:30.802177 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:30.802242 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:30.802324 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:30.803720 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.804459 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.806254 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.806813 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.806941 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:30.811899 (+ 4958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.194:36181: BlockingRecv error: recv got EOF from 127.11.192.194:36181 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:03:30.812834 28401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:30.799458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.194:36181)
0812 07:03:30.800212 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:30.800265 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:30.801938 (+ 1673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:30.802522 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:30.802529 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:30.803053 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:30.803589 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:30.803608 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.804614 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.804618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:30.806037 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:30.806056 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.807008 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.807014 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:30.807218 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:30.807836 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:30.807861 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:30.811728 (+ 3867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":273,"threads_started":1}
W20260812 07:03:30.813460 28398 heartbeater.cc:644] Failed to heartbeat to 127.11.192.254:39169 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.192.254:39169: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Aug 12 07:03:30 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:30 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:30.821189 28401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:30.813948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.194:35053)
0812 07:03:30.814101 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:30.814117 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:30.814200 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:30.814489 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:30.814493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:30.814823 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:30.814999 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:30.815010 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.815928 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.815933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:30.816997 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:30.817010 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.817829 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.817832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:30.817943 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:30.818435 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:30.818451 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:30.821013 (+ 2562us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 07:03:30.821501 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:30.814134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:35053 (local address 127.11.192.254:39169)
0812 07:03:30.814265 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:30.814269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:30.814283 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:03:30.814323 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:30.814326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:30.814375 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:30.814449 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:30.815127 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.815801 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.817127 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.817704 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.817836 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:30.821271 (+ 3435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.194:35053: BlockingRecv error: recv got EOF from 127.11.192.194:35053 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 07:03:30 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:30 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:30.828850 28401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:30.821755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.194:49967)
0812 07:03:30.821877 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:30.821894 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:30.821980 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:30.822252 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:30.822255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:30.822495 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:30.822620 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:30.822627 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.823770 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.823773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:30.824680 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:30.824691 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.825521 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.825526 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:30.825665 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:30.826140 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:30.826156 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:30.828660 (+ 2504us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 07:03:30.829103 28398 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 07:03:30.829128 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:30.821856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:49967 (local address 127.11.192.254:39169)
0812 07:03:30.822003 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:30.822008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:30.822048 (+ 40us) server_negotiation.cc:407] Connection header received
0812 07:03:30.822091 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:30.822095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:30.822151 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:30.822240 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:30.822752 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.823664 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.824810 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:30.825387 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:30.825518 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:30.828901 (+ 3383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.194:49967: BlockingRecv error: recv got EOF from 127.11.192.194:49967 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260812 07:03:31.128898 28404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:31.129149 28404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:31.129251 28404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:31.136608 28404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:31.136704 28404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:31.136816 28404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:03:31.146083 28404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/krb5kdc/oryx_127.11.192.195.keytab
--principal=oryx/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:42243
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=32929
--tserver_master_addrs=127.11.192.254:39169
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:31.147228 28404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:31.148443 28404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:31.160776 28413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:31.160815 28414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:31.160899 28416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:31.161420 28404 server_base.cc:1034] running on GCE node
I20260812 07:03:31.161998 28404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:31.162873 28404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:31.164059 28404 hybrid_clock.cc:648] HybridClock initialized: now 1786518211164017 us; error 34 us; skew 500 ppm
I20260812 07:03:31.164359 28404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:31 dist-test-slave-lgr5 krb5kdc[26810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518211, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:31.169864 28404 init.cc:377] Logged in from keytab as oryx/127.11.192.195@KRBTEST.COM (short username oryx)
I20260812 07:03:31.171162 28404 webserver.cc:460] Webserver started at http://127.11.192.195:32929/ using document root <none> and password file <none>
I20260812 07:03:31.171761 28404 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:31.171867 28404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:31.176867 28404 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260812 07:03:31.179358 28434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:31.180584 28404 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 07:03:31.180732 28404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "0670a15557d14b298f7142dc9bb913d2"
format_stamp: "Formatted at 2026-08-12 07:03:28 on dist-test-slave-lgr5"
server_key: "7163eaa4dcfc91cd115aa6b8ac670e7f"
server_key_iv: "5c27bc890f65758d4f77add1984d7259"
server_key_version: "encryptionkey@0"
I20260812 07:03:31.181617 28404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:31.192514 28404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:31.196343 28404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:31.196573 28404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:31.197412 28404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:31.198598 28404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:31.198688 28404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:31.198789 28404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:31.198859 28404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:31.227622 28404 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:42243
I20260812 07:03:31.227684 28650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:42243 every 8 connection(s)
I20260812 07:03:31.229002 28404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:03:31.233254 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 28404
Aug 12 07:03:31 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:31 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 07:03:31.244549 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:31.231319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:59959 (local address 127.11.192.254:39169)
0812 07:03:31.231519 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:31.231527 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:31.232609 (+ 1082us) server_negotiation.cc:407] Connection header received
0812 07:03:31.234911 (+ 2302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:31.234917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:31.235031 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:31.235167 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:31.236854 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:31.237630 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:31.239081 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:31.239556 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:31.239666 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:31.244241 (+ 4575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.195:59959: BlockingRecv error: recv got EOF from 127.11.192.195:59959 (error 108)
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":57,"spinlock_wait_cycles":6016}
I20260812 07:03:31.245162 28661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:31.231748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.195:59959)
0812 07:03:31.232453 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:31.232496 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:31.234607 (+ 2111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:31.235417 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:31.235429 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:31.236016 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:31.236686 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:31.236713 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:31.237766 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:31.237770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:31.238944 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:31.238956 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:31.239764 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:31.239772 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:31.239974 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:31.240566 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:31.240587 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:31.244021 (+ 3434us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":293,"threads_started":1}
W20260812 07:03:31.245761 28655 heartbeater.cc:644] Failed to heartbeat to 127.11.192.254:39169 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.192.254:39169: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
I20260812 07:03:31.248808 28667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:31.234276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52606 (local address 127.11.192.254:39169)
0812 07:03:31.235274 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:31.235280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:31.235295 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:03:31.235341 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:31.235345 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:31.235411 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:31.235516 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:31.237026 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:31.237837 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:31.239074 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:31.239503 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:31.239610 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:31.242297 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:31.242304 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:31.242317 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:31.242355 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:31.245510 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:31.246256 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:31.246265 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:31.246272 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:31.246500 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:31.246901 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:31.246905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:31.246908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:31.247367 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:31.247686 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:31.248038 (+ 352us) server_negotiation.cc:299] Negotiation successful
0812 07:03:31.248411 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":888,"thread_start_us":779,"threads_started":1}
Aug 12 07:03:31 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:31 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:31.254252 28661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:31.246299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.195:41555)
0812 07:03:31.246577 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:31.246595 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:31.246696 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:31.246943 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:31.246948 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:31.247217 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:31.247336 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:31.247342 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:31.248781 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:31.248785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:31.250028 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:31.250041 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:31.250909 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:31.250914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:31.251101 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:31.251625 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:31.251646 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:31.254075 (+ 2429us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 07:03:31.254403 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:31.246357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:41555 (local address 127.11.192.254:39169)
0812 07:03:31.246485 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:31.246490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:31.246644 (+ 154us) server_negotiation.cc:407] Connection header received
0812 07:03:31.246784 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:31.246786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:31.246832 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:31.246905 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:31.247460 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:31.248658 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:31.250156 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:31.250780 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:31.250917 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:31.254230 (+ 3313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.195:41555: BlockingRecv error: recv got EOF from 127.11.192.195:41555 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 12 07:03:31 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:31 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:31.261592 28661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:31.254782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.195:49033)
0812 07:03:31.254923 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:31.254940 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:31.255059 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:31.255421 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:31.255424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:31.255666 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:31.255795 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:31.255803 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:31.256758 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:31.256761 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:31.257670 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:31.257681 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:31.258308 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:31.258314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:31.258438 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:31.258848 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:31.258862 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:31.261303 (+ 2441us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 07:03:31.261747 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:31.254996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:49033 (local address 127.11.192.254:39169)
0812 07:03:31.255209 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:31.255213 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:31.255227 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:03:31.255261 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:31.255264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:31.255322 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:31.255392 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:31.255890 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:31.256663 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:31.257781 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:31.258195 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:31.258282 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:31.261523 (+ 3241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.195:49033: BlockingRecv error: recv got EOF from 127.11.192.195:49033 (error 108)
Metrics: {"server-negotiator.queue_time_us":131}
W20260812 07:03:31.261861 28655 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 07:03:31 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:31 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:31.410701 28669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:31.403379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.193:36259)
0812 07:03:31.403823 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:31.403839 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:31.403947 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:31.404219 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:31.404222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:31.404668 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:31.404857 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:31.404865 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:31.405749 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:31.405753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:31.406678 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:31.406689 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:31.407330 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:31.407334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:31.407444 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:31.407945 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:31.407961 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:31.410480 (+ 2519us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":226,"threads_started":1}
W20260812 07:03:31.410847 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:31.403437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:36259 (local address 127.11.192.254:39169)
0812 07:03:31.403641 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:31.403647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:31.403906 (+ 259us) server_negotiation.cc:407] Connection header received
0812 07:03:31.404037 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:31.404041 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:31.404120 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:31.404197 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:31.404959 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:31.405640 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:31.406777 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:31.407202 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:31.407289 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:31.410672 (+ 3383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.193:36259: BlockingRecv error: recv got EOF from 127.11.192.193:36259 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 12 07:03:31 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:31 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:31.837777 28670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:31.829620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.194:58805)
0812 07:03:31.830090 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:31.830107 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:31.830214 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:31.830466 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:31.830469 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:31.830907 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:31.831116 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:31.831126 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:31.832027 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:31.832030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:31.833133 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:31.833149 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:31.833804 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:31.833808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:31.833936 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:31.834475 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:31.834503 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:31.837480 (+ 2977us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":240,"threads_started":1}
W20260812 07:03:31.837932 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:31.829691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:58805 (local address 127.11.192.254:39169)
0812 07:03:31.829871 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:31.829876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:31.830169 (+ 293us) server_negotiation.cc:407] Connection header received
0812 07:03:31.830305 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:31.830308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:31.830368 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:31.830436 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:31.831223 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:31.831911 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:31.833262 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:31.833673 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:31.833761 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:31.837704 (+ 3943us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.194:58805: BlockingRecv error: recv got EOF from 127.11.192.194:58805 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 07:03:32 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:32 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:32.270030 28672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:32.262384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.195:57945)
0812 07:03:32.262893 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:32.262914 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:32.263030 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:32.263283 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:32.263287 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:32.263728 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:32.263913 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:32.263922 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:32.264869 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:32.264873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:32.265827 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:32.265838 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:32.266452 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:32.266457 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:32.266574 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:32.267127 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:32.267145 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:32.269740 (+ 2595us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":273,"threads_started":1}
W20260812 07:03:32.270201 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:32.262464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:57945 (local address 127.11.192.254:39169)
0812 07:03:32.262626 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:32.262630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:32.262977 (+ 347us) server_negotiation.cc:407] Connection header received
0812 07:03:32.263114 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:32.263118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:32.263177 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:32.263256 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:32.264030 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:32.264758 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:32.265932 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:32.266321 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:32.266406 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:32.269958 (+ 3552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.195:57945: BlockingRecv error: recv got EOF from 127.11.192.195:57945 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 07:03:32 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:32 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:32.419198 28673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:32.411560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.193:43297)
0812 07:03:32.412057 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:32.412072 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:32.412229 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:32.412534 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:32.412537 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:32.412931 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:32.413140 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:32.413155 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:32.414085 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:32.414089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:32.414966 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:32.414978 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:32.415588 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:32.415592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:32.415700 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:32.416242 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:32.416260 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:32.418962 (+ 2702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":261,"threads_started":1}
W20260812 07:03:32.419407 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:32.411616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:43297 (local address 127.11.192.254:39169)
0812 07:03:32.411764 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:32.411769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:32.412153 (+ 384us) server_negotiation.cc:407] Connection header received
0812 07:03:32.412340 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:32.412344 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:32.412405 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:32.412485 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:32.413255 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:32.413989 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:32.415079 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:32.415463 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:32.415554 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:32.419171 (+ 3617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.193:43297: BlockingRecv error: recv got EOF from 127.11.192.193:43297 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 07:03:32 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:32 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:32.847092 28674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:32.838671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.194:40121)
0812 07:03:32.839178 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:32.839194 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:32.839333 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:32.839658 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:32.839663 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:32.840105 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:32.840280 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:32.840291 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:32.841276 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:32.841282 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:32.842180 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:32.842193 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:32.842895 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:32.842901 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:32.843063 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:32.843616 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:32.843635 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:32.846757 (+ 3122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":298,"threads_started":1}
W20260812 07:03:32.847172 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:32.838764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:40121 (local address 127.11.192.254:39169)
0812 07:03:32.838956 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:32.838962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:32.839274 (+ 312us) server_negotiation.cc:407] Connection header received
0812 07:03:32.839448 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:32.839454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:32.839526 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:32.839606 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:32.840422 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:32.841135 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:32.842337 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:32.842744 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:32.842851 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:32.846960 (+ 4109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.194:40121: BlockingRecv error: recv got EOF from 127.11.192.194:40121 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 07:03:33 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:33 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:33.278499 28675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:33.270852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.195:35821)
0812 07:03:33.271456 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:33.271478 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:33.271596 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:33.271875 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:33.271878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:33.272328 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:33.272506 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:33.272515 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:33.273448 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:33.273452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:33.274327 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:33.274337 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:33.274947 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:33.274952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:33.275087 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:33.275578 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:33.275595 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:33.278256 (+ 2661us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":348,"threads_started":1}
W20260812 07:03:33.278666 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:33.271008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:35821 (local address 127.11.192.254:39169)
0812 07:03:33.271242 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:33.271248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:33.271549 (+ 301us) server_negotiation.cc:407] Connection header received
0812 07:03:33.271699 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:33.271702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:33.271776 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:33.271848 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:33.272624 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:33.273345 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:33.274424 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:33.274839 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:33.274926 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:33.278427 (+ 3501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.195:35821: BlockingRecv error: recv got EOF from 127.11.192.195:35821 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 12 07:03:33 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:33 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:33.427536 28676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:33.420185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.193:33851)
0812 07:03:33.420693 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:33.420709 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:33.420823 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:33.421098 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:33.421101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:33.421434 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:33.421606 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:33.421614 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:33.422570 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:33.422574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:33.423443 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:33.423454 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:33.424033 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:33.424036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:33.424173 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:33.424694 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:33.424710 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:33.427281 (+ 2571us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":261,"threads_started":1}
W20260812 07:03:33.427675 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:33.420223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:33851 (local address 127.11.192.254:39169)
0812 07:03:33.420391 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:33.420398 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:33.420790 (+ 392us) server_negotiation.cc:407] Connection header received
0812 07:03:33.420917 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:33.420920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:33.420992 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:33.421067 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:33.421714 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:33.422449 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:33.423544 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:33.423914 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:33.424005 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:33.427454 (+ 3449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.193:33851: BlockingRecv error: recv got EOF from 127.11.192.193:33851 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 07:03:33 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:33 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:33.855738 28677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:33.847956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.194:59881)
0812 07:03:33.848486 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:33.848502 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:33.848617 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:33.848870 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:33.848873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:33.849213 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:33.849386 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:33.849395 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:33.850357 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:33.850361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:33.851242 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:33.851253 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:33.851841 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:33.851845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:33.851963 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:33.852569 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:33.852588 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:33.855478 (+ 2890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":259,"threads_started":1}
W20260812 07:03:33.855863 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:33.848014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:59881 (local address 127.11.192.254:39169)
0812 07:03:33.848180 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:33.848184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:33.848565 (+ 381us) server_negotiation.cc:407] Connection header received
0812 07:03:33.848702 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:33.848706 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:33.848764 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:33.848841 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:33.849502 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:33.850246 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:33.851341 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:33.851738 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:33.851823 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:33.855655 (+ 3832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.194:59881: BlockingRecv error: recv got EOF from 127.11.192.194:59881 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 07:03:34 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:34 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:34.287693 28678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:34.279287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.195:40683)
0812 07:03:34.279787 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:34.279806 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:34.279933 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:34.280283 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:34.280289 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:34.280668 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:34.280857 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:34.280871 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:34.281921 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:34.281925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:34.283220 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:34.283237 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:34.283998 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:34.284003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:34.284174 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:34.284779 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:34.284797 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:34.287409 (+ 2612us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":250,"threads_started":1}
W20260812 07:03:34.287824 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:34.279369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:40683 (local address 127.11.192.254:39169)
0812 07:03:34.279558 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:34.279564 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:34.279886 (+ 322us) server_negotiation.cc:407] Connection header received
0812 07:03:34.280046 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:34.280051 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:34.280150 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:34.280230 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:34.281008 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:34.281746 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:34.283360 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:34.283860 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:34.283953 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:34.287596 (+ 3643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.195:40683: BlockingRecv error: recv got EOF from 127.11.192.195:40683 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 07:03:34 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:34 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:34.436424 28680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:34.428391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.193:48769)
0812 07:03:34.428867 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:34.428882 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:34.429026 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:34.429352 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:34.429356 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:34.429735 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:34.429888 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:34.429897 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:34.430906 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:34.430910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:34.431880 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:34.431892 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:34.432641 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:34.432646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:34.432787 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:34.433370 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:34.433388 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:34.436121 (+ 2733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":258,"threads_started":1}
W20260812 07:03:34.436589 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:34.428481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:48769 (local address 127.11.192.254:39169)
0812 07:03:34.428663 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:34.428670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:34.428965 (+ 295us) server_negotiation.cc:407] Connection header received
0812 07:03:34.429148 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:34.429152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:34.429211 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:34.429291 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:34.430028 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:34.430775 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:34.432024 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:34.432509 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:34.432634 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:34.436346 (+ 3712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.193:48769: BlockingRecv error: recv got EOF from 127.11.192.193:48769 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 07:03:34 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:34 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:34.864241 28681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:34.856542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.194:51175)
0812 07:03:34.857077 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:34.857093 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:34.857209 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:34.857497 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:34.857500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:34.857909 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:34.858071 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:34.858081 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:34.859052 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:34.859055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:34.860019 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:34.860031 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:34.860715 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:34.860719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:34.860844 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:34.861409 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:34.861447 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:34.863995 (+ 2548us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":266,"threads_started":1}
W20260812 07:03:34.864403 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:34.856596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:51175 (local address 127.11.192.254:39169)
0812 07:03:34.856746 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:34.856751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:34.857158 (+ 407us) server_negotiation.cc:407] Connection header received
0812 07:03:34.857312 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:34.857315 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:34.857376 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:34.857446 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:34.858191 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:34.858941 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:34.860167 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:34.860591 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:34.860676 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:34.864206 (+ 3530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.194:51175: BlockingRecv error: recv got EOF from 127.11.192.194:51175 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 12 07:03:35 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:35 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:35.296845 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:35.288523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.195:33057)
0812 07:03:35.288997 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:35.289015 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:35.289144 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:35.289462 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:35.289468 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:35.289876 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:35.290028 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:35.290037 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:35.291297 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:35.291303 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:35.292424 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:35.292438 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:35.293112 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:35.293118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:35.293283 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:35.293841 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:35.293858 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:35.296534 (+ 2676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":265,"threads_started":1}
W20260812 07:03:35.296985 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:35.288587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:33057 (local address 127.11.192.254:39169)
0812 07:03:35.288755 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:35.288762 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:35.289097 (+ 335us) server_negotiation.cc:407] Connection header received
0812 07:03:35.289268 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:35.289272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:35.289334 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:35.289413 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:35.290174 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:35.291177 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:35.292564 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:35.292973 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:35.293086 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:35.296741 (+ 3655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.195:33057: BlockingRecv error: recv got EOF from 127.11.192.195:33057 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 07:03:35 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:35 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:35.445485 28686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:35.437336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.193:33261)
0812 07:03:35.437868 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:35.437884 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:35.438002 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:35.438263 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:35.438266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:35.438766 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:35.438940 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:35.438950 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:35.439888 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:35.439891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:35.440936 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:35.440954 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:35.441793 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:35.441799 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:35.441959 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:35.442489 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:35.442509 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:35.445229 (+ 2720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":261,"threads_started":1}
W20260812 07:03:35.445596 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:35.437424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:33261 (local address 127.11.192.254:39169)
0812 07:03:35.437596 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:35.437601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:35.437948 (+ 347us) server_negotiation.cc:407] Connection header received
0812 07:03:35.438093 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:35.438096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:35.438154 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:35.438233 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:35.439086 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:35.439790 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:35.441084 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:35.441632 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:35.441748 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:35.445413 (+ 3665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.193:33261: BlockingRecv error: recv got EOF from 127.11.192.193:33261 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 12 07:03:35 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:35 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:35.873270 28687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:35.865129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.194:46185)
0812 07:03:35.865651 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:35.865667 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:35.865779 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:35.866048 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:35.866052 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:35.866474 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:35.866644 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:35.866655 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:35.867788 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:35.867794 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:35.868982 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:35.868998 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:35.869625 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:35.869629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:35.869743 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:35.870310 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:35.870328 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:35.873012 (+ 2684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":250,"threads_started":1}
W20260812 07:03:35.873404 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:35.865294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:46185 (local address 127.11.192.254:39169)
0812 07:03:35.865494 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:35.865498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:35.865731 (+ 233us) server_negotiation.cc:407] Connection header received
0812 07:03:35.865869 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:35.865872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:35.865931 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:35.866011 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:35.866763 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:35.867650 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:35.869104 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:35.869516 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:35.869608 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:35.873188 (+ 3580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.194:46185: BlockingRecv error: recv got EOF from 127.11.192.194:46185 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 12 07:03:36 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:36 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:36.309494 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:36.297797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.195:33065)
0812 07:03:36.298378 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:36.298406 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:36.298547 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:36.298923 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:36.298929 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:36.299380 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:36.299542 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:36.299551 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:36.300603 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:36.300608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:36.301532 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:36.301543 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:36.302165 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:36.302169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:36.302279 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:36.306594 (+ 4315us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:36.306611 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:36.309160 (+ 2549us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":252,"threads_started":1}
W20260812 07:03:36.309580 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:36.297872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:33065 (local address 127.11.192.254:39169)
0812 07:03:36.298100 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:36.298106 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:36.298493 (+ 387us) server_negotiation.cc:407] Connection header received
0812 07:03:36.298679 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:36.298684 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:36.298762 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:36.298861 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:36.299664 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:36.300464 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:36.301654 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:36.302063 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:36.302156 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:36.309353 (+ 7197us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.195:33065: BlockingRecv error: recv got EOF from 127.11.192.195:33065 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 12 07:03:36 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:36 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:36.454259 28693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:36.446289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.193:47555)
0812 07:03:36.446769 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:36.446785 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:36.446917 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:36.447242 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:36.447247 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:36.447653 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:36.447848 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:36.447862 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:36.449207 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:36.449212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:36.450107 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:36.450119 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:36.450739 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:36.450743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:36.450895 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:36.451418 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:36.451436 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:36.454036 (+ 2600us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":255,"threads_started":1}
W20260812 07:03:36.454414 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:36.446339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:47555 (local address 127.11.192.254:39169)
0812 07:03:36.446515 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:36.446521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:36.446871 (+ 350us) server_negotiation.cc:407] Connection header received
0812 07:03:36.447046 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:36.447049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:36.447109 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:36.447186 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:36.448003 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:36.449082 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:36.450220 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:36.450628 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:36.450714 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:36.454201 (+ 3487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.193:47555: BlockingRecv error: recv got EOF from 127.11.192.193:47555 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 07:03:36 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:36 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:36.882081 28694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:36.874143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.194:59075)
0812 07:03:36.874671 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:36.874687 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:36.874809 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:36.875093 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:36.875096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:36.875496 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:36.875661 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:36.875670 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:36.876786 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:36.876790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:36.877685 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:36.877699 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:36.878311 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:36.878315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:36.878436 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:36.879002 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:36.879021 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:36.881773 (+ 2752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":287,"threads_started":1}
W20260812 07:03:36.882334 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:36.874207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:59075 (local address 127.11.192.254:39169)
0812 07:03:36.874375 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:36.874379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:36.874750 (+ 371us) server_negotiation.cc:407] Connection header received
0812 07:03:36.874908 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:36.874912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:36.874978 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:36.875059 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:36.875771 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:36.876658 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:36.877799 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:36.878192 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:36.878278 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:36.882072 (+ 3794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.194:59075: BlockingRecv error: recv got EOF from 127.11.192.194:59075 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 07:03:37 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:37 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:37.318230 28695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:37.310384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.195:50259)
0812 07:03:37.310888 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:37.310907 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:37.311037 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:37.311351 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:37.311356 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:37.311761 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:37.311912 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:37.311921 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:37.313000 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:37.313005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:37.313892 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:37.313904 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:37.314622 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:37.314628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:37.314793 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:37.315320 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:37.315337 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:37.317947 (+ 2610us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":286,"threads_started":1}
W20260812 07:03:37.318406 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:37.310499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:50259 (local address 127.11.192.254:39169)
0812 07:03:37.310691 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:37.310697 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:37.310995 (+ 298us) server_negotiation.cc:407] Connection header received
0812 07:03:37.311145 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:37.311149 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:37.311221 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:37.311303 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:37.312097 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:37.312854 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:37.314040 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:37.314470 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:37.314583 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:37.318156 (+ 3573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.195:50259: BlockingRecv error: recv got EOF from 127.11.192.195:50259 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 07:03:37 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:37 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:37.463265 28699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:37.455091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.193:58883)
0812 07:03:37.455627 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:37.455642 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:37.455753 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:37.456024 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:37.456027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:37.456473 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:37.456683 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:37.456698 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:37.458084 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:37.458088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:37.458976 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:37.458998 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:37.459612 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:37.459616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:37.459724 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:37.460285 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:37.460303 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:37.463011 (+ 2708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":228,"threads_started":1}
W20260812 07:03:37.463382 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:37.455166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:58883 (local address 127.11.192.254:39169)
0812 07:03:37.455356 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:37.455363 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:37.455716 (+ 353us) server_negotiation.cc:407] Connection header received
0812 07:03:37.455838 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:37.455841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:37.455901 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:37.455983 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:37.456826 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:37.457924 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:37.459096 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:37.459502 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:37.459589 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:37.463203 (+ 3614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.193:58883: BlockingRecv error: recv got EOF from 127.11.192.193:58883 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 07:03:37 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:37 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:37.891060 28700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:37.882936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.194:47699)
0812 07:03:37.883548 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:37.883565 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:37.883680 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:37.883963 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:37.883966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:37.884396 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:37.884574 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:37.884584 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:37.885662 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:37.885666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:37.886576 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:37.886589 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:37.887364 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:37.887370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:37.887536 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:37.888118 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:37.888137 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:37.890673 (+ 2536us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":266,"threads_started":1}
W20260812 07:03:37.891117 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:37.883007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:47699 (local address 127.11.192.254:39169)
0812 07:03:37.883174 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:37.883179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:37.883627 (+ 448us) server_negotiation.cc:407] Connection header received
0812 07:03:37.883772 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:37.883775 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:37.883847 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:37.883921 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:37.884740 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:37.885518 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:37.886713 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:37.887208 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:37.887321 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:37.890888 (+ 3567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.194:47699: BlockingRecv error: recv got EOF from 127.11.192.194:47699 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 12 07:03:38 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:38 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:38.326586 28705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:38.319096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.195:38647)
0812 07:03:38.319662 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:38.319681 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:38.319794 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:38.320062 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:38.320066 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:38.320519 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:38.320691 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:38.320700 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:38.321607 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:38.321611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:38.322511 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:38.322522 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:38.323128 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:38.323132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:38.323239 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:38.323758 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:38.323773 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:38.326339 (+ 2566us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":238,"threads_started":1}
W20260812 07:03:38.326756 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:38.319168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:38647 (local address 127.11.192.254:39169)
0812 07:03:38.319330 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:38.319335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:38.319744 (+ 409us) server_negotiation.cc:407] Connection header received
0812 07:03:38.319877 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:38.319880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:38.319953 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:38.320033 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:38.320801 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:38.321503 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:38.322613 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:38.323019 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:38.323108 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:38.326533 (+ 3425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.195:38647: BlockingRecv error: recv got EOF from 127.11.192.195:38647 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 07:03:38 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:38 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:38.472595 28706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:38.464128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.193:54887)
0812 07:03:38.464645 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:38.464662 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:38.464777 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:38.465039 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:38.465042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:38.465440 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:38.465609 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:38.465618 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:38.466689 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:38.466694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:38.468118 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:38.468138 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:38.469016 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:38.469024 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:38.469181 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:38.469704 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:38.469721 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:38.472288 (+ 2567us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":246,"threads_started":1}
W20260812 07:03:38.472738 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:38.464261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:54887 (local address 127.11.192.254:39169)
0812 07:03:38.464421 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:38.464425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:38.464726 (+ 301us) server_negotiation.cc:407] Connection header received
0812 07:03:38.464867 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:38.464870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:38.464930 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:38.465011 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:38.465716 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:38.466531 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:38.468261 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:38.468814 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:38.468926 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:38.472481 (+ 3555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.193:54887: BlockingRecv error: recv got EOF from 127.11.192.193:54887 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 07:03:38 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:38 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:38.900187 28707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:38.892036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.194:35041)
0812 07:03:38.892603 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:38.892620 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:38.892741 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:38.892987 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:38.892990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:38.893400 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:38.893573 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:38.893583 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:38.894796 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:38.894800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:38.895750 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:38.895763 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:38.896533 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:38.896539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:38.896657 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:38.897194 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:38.897211 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:38.899908 (+ 2697us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":268,"threads_started":1}
W20260812 07:03:38.900329 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:38.892110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:35041 (local address 127.11.192.254:39169)
0812 07:03:38.892253 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:38.892258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:38.892686 (+ 428us) server_negotiation.cc:407] Connection header received
0812 07:03:38.892819 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:38.892822 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:38.892880 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:38.892957 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:38.893730 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:38.894667 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:38.895874 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:38.896360 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:38.896455 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:38.900117 (+ 3662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.194:35041: BlockingRecv error: recv got EOF from 127.11.192.194:35041 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 12 07:03:39 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:39 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:39.335218 28708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:39.327438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.195:57657)
0812 07:03:39.327928 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:39.327948 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:39.328059 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:39.328345 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:39.328349 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:39.328750 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:39.328899 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:39.328907 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:39.329848 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:39.329852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:39.330917 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:39.330930 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:39.331589 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:39.331594 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:39.331704 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:39.332326 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:39.332343 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:39.334977 (+ 2634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":253,"threads_started":1}
W20260812 07:03:39.335413 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:39.327493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:57657 (local address 127.11.192.254:39169)
0812 07:03:39.327734 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:39.327741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:39.328021 (+ 280us) server_negotiation.cc:407] Connection header received
0812 07:03:39.328164 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:39.328168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:39.328241 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:39.328341 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:39.329003 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:39.329739 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:39.331029 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:39.331447 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:39.331534 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:39.335179 (+ 3645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.195:57657: BlockingRecv error: recv got EOF from 127.11.192.195:57657 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 12 07:03:39 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:39 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:39.481797 28710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:39.473529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.193:57241)
0812 07:03:39.474033 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:39.474054 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:39.474202 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:39.474568 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:39.474573 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:39.474981 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:39.475177 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:39.475187 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:39.476332 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:39.476336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:39.477270 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:39.477283 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:39.478019 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:39.478026 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:39.478178 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:39.478734 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:39.478752 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:39.481557 (+ 2805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":264,"threads_started":1}
W20260812 07:03:39.481928 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:39.473594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:57241 (local address 127.11.192.254:39169)
0812 07:03:39.473765 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:39.473771 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:39.474143 (+ 372us) server_negotiation.cc:407] Connection header received
0812 07:03:39.474338 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:39.474343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:39.474421 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:39.474521 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:39.475322 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:39.476203 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:39.477409 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:39.477859 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:39.477964 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:39.481731 (+ 3767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.193:57241: BlockingRecv error: recv got EOF from 127.11.192.193:57241 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 07:03:39 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:39 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:39.909832 28711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:39.901137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.194:53013)
0812 07:03:39.901588 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:39.901604 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:39.901736 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:39.902042 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:39.902047 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:39.902455 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:39.902626 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:39.902635 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:39.903649 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:39.903654 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:39.904608 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:39.904621 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:39.905359 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:39.905366 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:39.905532 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:39.906084 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:39.906103 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:39.909493 (+ 3390us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":225,"threads_started":1}
W20260812 07:03:39.909935 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:39.901197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:53013 (local address 127.11.192.254:39169)
0812 07:03:39.901379 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:39.901385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:39.901687 (+ 302us) server_negotiation.cc:407] Connection header received
0812 07:03:39.901847 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:39.901850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:39.901910 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:39.901987 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:39.902773 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:39.903516 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:39.904747 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:39.905203 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:39.905324 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:39.909688 (+ 4364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.194:53013: BlockingRecv error: recv got EOF from 127.11.192.194:53013 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 07:03:40 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:40 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:40.344064 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:40.336033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.195:43971)
0812 07:03:40.336558 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:40.336578 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:40.336711 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:40.337076 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:40.337082 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:40.337481 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:40.337646 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:40.337655 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:40.338644 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:40.338650 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:40.339618 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:40.339630 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:40.340415 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:40.340422 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:40.340578 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:40.341082 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:40.341099 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:40.343823 (+ 2724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":259,"threads_started":1}
W20260812 07:03:40.344288 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:40.336157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:43971 (local address 127.11.192.254:39169)
0812 07:03:40.336350 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:40.336357 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:40.336661 (+ 304us) server_negotiation.cc:407] Connection header received
0812 07:03:40.336823 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:40.336828 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:40.336904 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:40.336985 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:40.337791 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:40.338515 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:40.339758 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:40.340237 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:40.340346 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:40.344009 (+ 3663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.195:43971: BlockingRecv error: recv got EOF from 127.11.192.195:43971 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 07:03:40 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:40 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:40.490795 28713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:40.482675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.193:37481)
0812 07:03:40.483199 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:40.483219 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:40.483350 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:40.483686 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:40.483690 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:40.484140 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:40.484344 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:40.484356 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:40.485403 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:40.485409 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:40.486355 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:40.486369 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:40.487060 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:40.487064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:40.487178 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:40.487754 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:40.487773 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:40.490496 (+ 2723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":303,"threads_started":1}
W20260812 07:03:40.490969 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:40.482754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:37481 (local address 127.11.192.254:39169)
0812 07:03:40.482936 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:40.482942 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:40.483303 (+ 361us) server_negotiation.cc:407] Connection header received
0812 07:03:40.483482 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:40.483486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:40.483553 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:40.483632 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:40.484494 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:40.485263 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:40.486479 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:40.486936 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:40.487031 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:40.490746 (+ 3715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.193:37481: BlockingRecv error: recv got EOF from 127.11.192.193:37481 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 07:03:40 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:40 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:40.919077 28714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:40.910893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.194:60589)
0812 07:03:40.911383 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:40.911402 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:40.911534 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:40.911832 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:40.911837 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:40.912349 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:40.912545 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:40.912557 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:40.913568 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:40.913574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:40.914905 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:40.914919 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:40.915581 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:40.915585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:40.915694 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:40.916283 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:40.916301 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:40.918819 (+ 2518us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":253,"threads_started":1}
W20260812 07:03:40.919307 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:40.910947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:60589 (local address 127.11.192.254:39169)
0812 07:03:40.911150 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:40.911156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:40.911482 (+ 326us) server_negotiation.cc:407] Connection header received
0812 07:03:40.911635 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:40.911638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:40.911699 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:40.911780 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:40.912683 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:40.913432 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:40.915036 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:40.915465 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:40.915558 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:40.919059 (+ 3501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.194:60589: BlockingRecv error: recv got EOF from 127.11.192.194:60589 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 07:03:41 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:41 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:41.355310 28716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:41.345038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.195:47833)
0812 07:03:41.345666 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:41.345687 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:41.345796 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:41.346074 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:41.346078 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:41.346426 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:41.346582 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:41.346590 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:41.347516 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:41.347520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:41.348488 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:41.348502 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:41.349115 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:41.349119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:41.349227 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:41.352471 (+ 3244us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:41.352490 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:41.355086 (+ 2596us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":379,"threads_started":1}
W20260812 07:03:41.355497 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:41.345268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:47833 (local address 127.11.192.254:39169)
0812 07:03:41.345531 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:41.345537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:41.345758 (+ 221us) server_negotiation.cc:407] Connection header received
0812 07:03:41.345881 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:41.345885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:41.345968 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:41.346047 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:41.346689 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:41.347401 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:41.348592 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:41.349003 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:41.349087 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:41.355265 (+ 6178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.195:47833: BlockingRecv error: recv got EOF from 127.11.192.195:47833 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
Aug 12 07:03:41 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:41 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:41.500126 28717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:41.491651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.193:40941)
0812 07:03:41.492176 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:41.492192 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:41.492322 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:41.492677 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:41.492682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:41.493130 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:41.493311 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:41.493320 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:41.494525 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:41.494531 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:41.495443 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:41.495456 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:41.496209 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:41.496216 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:41.496390 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:41.496930 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:41.496949 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:41.499769 (+ 2820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":261,"threads_started":1}
W20260812 07:03:41.500269 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:41.491754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:40941 (local address 127.11.192.254:39169)
0812 07:03:41.491936 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:41.491942 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:41.492270 (+ 328us) server_negotiation.cc:407] Connection header received
0812 07:03:41.492464 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:41.492467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:41.492540 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:41.492620 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:41.493440 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:41.494393 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:41.495588 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:41.496040 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:41.496186 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:41.500019 (+ 3833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.193:40941: BlockingRecv error: recv got EOF from 127.11.192.193:40941 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 07:03:41 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:41 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:41.927564 28719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:41.920153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.194:46403)
0812 07:03:41.920585 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:41.920603 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:41.920714 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:41.920978 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:41.920980 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:41.921390 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:41.921575 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:41.921584 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:41.922489 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:41.922492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:41.923368 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:41.923383 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:41.923978 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:41.923982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:41.924149 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:41.924744 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:41.924763 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:41.927336 (+ 2573us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":215,"threads_started":1}
W20260812 07:03:41.927739 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:41.920226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:46403 (local address 127.11.192.254:39169)
0812 07:03:41.920422 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:41.920428 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:41.920669 (+ 241us) server_negotiation.cc:407] Connection header received
0812 07:03:41.920812 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:41.920815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:41.920877 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:41.920946 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:41.921678 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:41.922385 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:41.923478 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:41.923871 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:41.923956 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:41.927529 (+ 3573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.194:46403: BlockingRecv error: recv got EOF from 127.11.192.194:46403 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 12 07:03:42 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:42 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:42.364125 28720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:42.356130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.195:33133)
0812 07:03:42.356603 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:42.356622 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:42.356752 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:42.357081 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:42.357087 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:42.357562 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:42.357740 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:42.357749 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:42.358806 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:42.358812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:42.359704 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:42.359716 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:42.360448 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:42.360454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:42.360636 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:42.361196 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:42.361213 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:42.363859 (+ 2646us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":260,"threads_started":1}
W20260812 07:03:42.364295 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:42.356201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:33133 (local address 127.11.192.254:39169)
0812 07:03:42.356369 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:42.356375 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:42.356701 (+ 326us) server_negotiation.cc:407] Connection header received
0812 07:03:42.356875 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:42.356880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:42.356954 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:42.357034 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:42.357905 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:42.358684 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:42.359841 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:42.360302 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:42.360410 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:42.364051 (+ 3641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.195:33133: BlockingRecv error: recv got EOF from 127.11.192.195:33133 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 07:03:42 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:42 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:42.509810 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:42.501100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.193:33913)
0812 07:03:42.501593 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:42.501608 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:42.501736 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:42.502069 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:42.502074 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:42.502566 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:42.502768 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:42.502780 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:42.503933 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:42.503939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:42.505129 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:42.505144 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:42.505988 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:42.505993 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:42.506124 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:42.506715 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:42.506735 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:42.509564 (+ 2829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":272,"threads_started":1}
W20260812 07:03:42.509946 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:42.501176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:33913 (local address 127.11.192.254:39169)
0812 07:03:42.501378 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:42.501385 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:42.501687 (+ 302us) server_negotiation.cc:407] Connection header received
0812 07:03:42.501853 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:42.501858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:42.501926 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:42.502021 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:42.502903 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:42.503774 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:42.505288 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:42.505825 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:42.505949 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:42.509756 (+ 3807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.193:33913: BlockingRecv error: recv got EOF from 127.11.192.193:33913 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 12 07:03:42 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:42 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:42.936183 28726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:42.928388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.194:46551)
0812 07:03:42.928844 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:42.928860 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:42.928994 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:42.929301 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:42.929306 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:42.929745 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:42.929903 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:42.929913 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:42.930886 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:42.930890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:42.931820 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:42.931834 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:42.932566 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:42.932571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:42.932750 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:42.933277 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:42.933295 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:42.935836 (+ 2541us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":245,"threads_started":1}
W20260812 07:03:42.936326 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:42.928433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:46551 (local address 127.11.192.254:39169)
0812 07:03:42.928647 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:42.928653 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:42.928943 (+ 290us) server_negotiation.cc:407] Connection header received
0812 07:03:42.929107 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:42.929111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:42.929171 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:42.929251 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:42.930049 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:42.930745 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:42.931961 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:42.932424 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:42.932540 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:42.936108 (+ 3568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.194:46551: BlockingRecv error: recv got EOF from 127.11.192.194:46551 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 12 07:03:43 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:43 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:43.373288 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:43.364980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.195:34039)
0812 07:03:43.365462 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:43.365483 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:43.365605 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:43.365881 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:43.365885 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:43.366274 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:43.366436 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:43.366447 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:43.367458 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:43.367464 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:43.368960 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:43.368976 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:43.369626 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:43.369632 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:43.369769 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:43.370287 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:43.370303 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:43.372963 (+ 2660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":261,"threads_started":1}
W20260812 07:03:43.373440 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:43.365049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:34039 (local address 127.11.192.254:39169)
0812 07:03:43.365216 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:43.365220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:43.365553 (+ 333us) server_negotiation.cc:407] Connection header received
0812 07:03:43.365706 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:43.365709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:43.365779 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:43.365853 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:43.366595 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:43.367307 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:43.369077 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:43.369487 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:43.369574 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:43.373194 (+ 3620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.195:34039: BlockingRecv error: recv got EOF from 127.11.192.195:34039 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 07:03:43 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:43 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:43.518833 28735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:43.510668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.193:40689)
0812 07:03:43.511191 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:43.511207 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:43.511337 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:43.511721 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:43.511726 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:43.512233 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:43.512414 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:43.512425 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:43.513445 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:43.513449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:43.514367 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:43.514380 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:43.515133 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:43.515140 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:43.515295 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:43.515851 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:43.515871 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:43.518505 (+ 2634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":266,"threads_started":1}
W20260812 07:03:43.519089 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:43.510748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:40689 (local address 127.11.192.254:39169)
0812 07:03:43.510929 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:43.510935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:43.511289 (+ 354us) server_negotiation.cc:407] Connection header received
0812 07:03:43.511484 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:43.511488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:43.511548 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:43.511647 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:43.512564 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:43.513302 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:43.514520 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:43.514942 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:43.515067 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:43.518734 (+ 3667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.193:40689: BlockingRecv error: recv got EOF from 127.11.192.193:40689 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 07:03:43 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:43 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:43.945501 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:43.937215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.194:34521)
0812 07:03:43.937789 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:43.937807 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:43.937938 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:43.938256 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:43.938262 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:43.938796 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:43.938969 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:43.938980 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:43.939985 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:43.939991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:43.940964 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:43.940978 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:43.941687 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:43.941694 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:43.941854 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:43.942442 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:43.942461 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:43.945166 (+ 2705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":295,"threads_started":1}
W20260812 07:03:43.945737 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:43.937413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:34521 (local address 127.11.192.254:39169)
0812 07:03:43.937596 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:43.937602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:43.937890 (+ 288us) server_negotiation.cc:407] Connection header received
0812 07:03:43.938057 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:43.938061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:43.938123 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:43.938201 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:43.939111 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:43.939847 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:43.941108 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:43.941516 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:43.941625 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:43.945393 (+ 3768us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.194:34521: BlockingRecv error: recv got EOF from 127.11.192.194:34521 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 12 07:03:44 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:44 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:44.382277 28738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:44.374312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.195:34291)
0812 07:03:44.374813 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:44.374832 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:44.374962 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:44.375309 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:44.375315 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:44.375733 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:44.375902 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:44.375911 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:44.376935 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:44.376940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:44.377854 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:44.377867 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:44.378607 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:44.378615 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:44.378754 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:44.379294 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:44.379311 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:44.381960 (+ 2649us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":250,"threads_started":1}
W20260812 07:03:44.382424 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:44.374369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:34291 (local address 127.11.192.254:39169)
0812 07:03:44.374566 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:44.374572 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:44.374918 (+ 346us) server_negotiation.cc:407] Connection header received
0812 07:03:44.375089 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:44.375094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:44.375168 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:44.375253 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:44.376066 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:44.376795 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:44.378005 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:44.378420 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:44.378528 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:44.382191 (+ 3663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.195:34291: BlockingRecv error: recv got EOF from 127.11.192.195:34291 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 07:03:44 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:44 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:44.527599 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:44.519690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.193:47493)
0812 07:03:44.520203 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:44.520219 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:44.520350 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:44.520681 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:44.520687 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:44.521095 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:44.521276 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:44.521286 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:44.522394 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:44.522398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:44.523310 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:44.523322 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:44.524021 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:44.524027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:44.524213 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:44.524737 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:44.524754 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:44.527246 (+ 2492us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":244,"threads_started":1}
W20260812 07:03:44.527709 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:44.519774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:47493 (local address 127.11.192.254:39169)
0812 07:03:44.520072 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:44.520094 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:44.520293 (+ 199us) server_negotiation.cc:407] Connection header received
0812 07:03:44.520486 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:44.520490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:44.520550 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:44.520626 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:44.521415 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:44.522259 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:44.523448 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:44.523872 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:44.523978 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:44.527488 (+ 3510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.193:47493: BlockingRecv error: recv got EOF from 127.11.192.193:47493 (error 108)
Metrics: {"server-negotiator.queue_time_us":158}
Aug 12 07:03:44 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:44 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:44.953910 28741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:44.946310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.194:38121)
0812 07:03:44.946844 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:44.946860 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:44.947009 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:44.947380 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:44.947384 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:44.947759 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:44.947937 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:44.947946 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:44.948879 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:44.948883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:44.949780 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:44.949790 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:44.950389 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:44.950393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:44.950504 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:44.951047 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:44.951065 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:44.953653 (+ 2588us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":299,"threads_started":1}
W20260812 07:03:44.954025 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:44.946368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:38121 (local address 127.11.192.254:39169)
0812 07:03:44.946561 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:44.946567 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:44.946928 (+ 361us) server_negotiation.cc:407] Connection header received
0812 07:03:44.947147 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:44.947152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:44.947232 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:44.947329 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:44.948059 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:44.948778 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:44.949878 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:44.950285 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:44.950375 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:44.953824 (+ 3449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.194:38121: BlockingRecv error: recv got EOF from 127.11.192.194:38121 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 07:03:45 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:45 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.195@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:45.390897 28743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:45.383204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.195:51939)
0812 07:03:45.383678 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:45.383699 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:45.383812 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:45.384128 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:45.384133 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:45.384484 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:45.384674 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:45.384683 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:45.385653 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:45.385657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:45.386630 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:45.386644 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:45.387283 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:45.387287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:45.387438 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:45.387975 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:45.387993 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:45.390647 (+ 2654us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":247,"threads_started":1}
W20260812 07:03:45.391047 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:45.383293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:51939 (local address 127.11.192.254:39169)
0812 07:03:45.383462 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:45.383467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:45.383768 (+ 301us) server_negotiation.cc:407] Connection header received
0812 07:03:45.383893 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:45.383896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:45.383986 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:45.384063 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:45.384794 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:45.385538 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:45.386749 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:45.387178 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:45.387268 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:45.390844 (+ 3576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.195:51939: BlockingRecv error: recv got EOF from 127.11.192.195:51939 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 07:03:45 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:45 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.193@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:45.536605 28744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:45.528725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.193:55935)
0812 07:03:45.529175 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:45.529190 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:45.529307 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:45.529579 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:45.529582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:45.529985 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:45.530153 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:45.530161 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:45.531110 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:45.531114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:45.532062 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:45.532095 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:45.532931 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:45.532936 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:45.533058 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:45.533645 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:45.533664 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:45.536351 (+ 2687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":261,"threads_started":1}
W20260812 07:03:45.536784 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:45.528784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:55935 (local address 127.11.192.254:39169)
0812 07:03:45.528982 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:45.528988 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:45.529261 (+ 273us) server_negotiation.cc:407] Connection header received
0812 07:03:45.529402 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:45.529405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:45.529466 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:45.529551 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:45.530253 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:45.530953 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:45.532197 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:45.532789 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:45.532887 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:45.536556 (+ 3669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.193:55935: BlockingRecv error: recv got EOF from 127.11.192.193:55935 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 07:03:45 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:03:45 dist-test-slave-lgr5 krb5kdc[26810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.192.194@KRBTEST.COM for oryx/127.11.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:03:45.967159 28745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:45.954897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39169 (local address 127.11.192.194:37893)
0812 07:03:45.955416 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:45.955435 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:45.955601 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:45.955967 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:45.955972 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:45.956529 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:45.956714 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:45.956724 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:45.957713 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:45.957717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:45.958704 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:45.958717 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:45.959389 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:45.959392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:45.959509 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:45.964215 (+ 4706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:45.964234 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:45.966797 (+ 2563us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.192.254:39169: Server oryx/127.11.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":290,"threads_started":1}
W20260812 07:03:45.967254 28144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:45.954977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:37893 (local address 127.11.192.254:39169)
0812 07:03:45.955199 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:45.955206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:45.955518 (+ 312us) server_negotiation.cc:407] Connection header received
0812 07:03:45.955736 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:45.955741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:45.955821 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:45.955905 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:45.956821 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:45.957588 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:45.958826 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:45.959284 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:45.959383 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:45.967005 (+ 7622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.192.194:37893: BlockingRecv error: recv got EOF from 127.11.192.194:37893 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 07:03:46.237259 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 27879
I20260812 07:03:46.248739 28747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:46.238190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47728 (local address 127.11.192.193:42407)
0812 07:03:46.238739 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:46.238746 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:46.238774 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:03:46.238839 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:46.238846 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:46.239077 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:46.239240 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:46.240629 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:46.241682 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:46.242997 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:46.243539 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:46.243657 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:46.246009 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:46.246019 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:46.246032 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:46.246062 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:46.248277 (+ 2215us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.192.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 07:03:46.248522 (+ 245us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47728: Request ticket server kudu/127.11.192.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":289,"threads_started":1}
W20260812 07:03:46.248838 28747 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47728: Request ticket server kudu/127.11.192.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 07:03:46.248865 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 27879: Not authorized: Client connection negotiation failed: client connection to 127.11.192.193:42407: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.192.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 07:03:46.249073 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 27879
I20260812 07:03:46.275170 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 28155
I20260812 07:03:46.286392 28748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:46.276207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52040 (local address 127.11.192.194:38645)
0812 07:03:46.276775 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:46.276782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:46.276812 (+ 30us) server_negotiation.cc:407] Connection header received
0812 07:03:46.276873 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:46.276879 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:46.277094 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:46.277238 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:46.278383 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:46.279407 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:46.280632 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:46.281153 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:46.281299 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:46.283582 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:46.283591 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:46.283604 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:46.283633 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:46.285869 (+ 2236us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.192.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 07:03:46.286139 (+ 270us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52040: Request ticket server kudu/127.11.192.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":257,"threads_started":1}
W20260812 07:03:46.286437 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 28155: Not authorized: Client connection negotiation failed: client connection to 127.11.192.194:38645: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.192.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 07:03:46.286496 28748 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52040: Request ticket server kudu/127.11.192.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 07:03:46.286593 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 28155
I20260812 07:03:46.308252 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 28404
I20260812 07:03:46.319082 28750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:46.309147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35360 (local address 127.11.192.195:42243)
0812 07:03:46.309690 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:46.309697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:46.309721 (+ 24us) server_negotiation.cc:407] Connection header received
0812 07:03:46.309784 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:46.309790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:46.309988 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:46.310140 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:46.311314 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:46.312413 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:46.313477 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:46.314002 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:46.314094 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:46.316427 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:46.316435 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:46.316447 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:46.316477 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:46.318609 (+ 2132us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.192.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 07:03:46.318837 (+ 228us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35360: Request ticket server kudu/127.11.192.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":261,"threads_started":1}
W20260812 07:03:46.319177 28750 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35360: Request ticket server kudu/127.11.192.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 07:03:46.319219 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 28404: Not authorized: Client connection negotiation failed: client connection to 127.11.192.195:42243: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.11.192.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 07:03:46.319379 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 28404
I20260812 07:03:46.344004 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 27748
I20260812 07:03:46.345194 27864 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:46.462641 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 27748
2026-08-12T07:03:46Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21016 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T07:03:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:03:46Z Disabled control of system clock
I20260812 07:03:46.546813 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:41321
--webserver_interface=127.11.192.254
--webserver_port=0
--builtin_ntp_servers=127.11.192.212:41149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:41321
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 07:03:46.866247 28760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:46.866501 28760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:46.866581 28760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:46.873391 28760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:03:46.873479 28760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:46.873521 28760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:46.873576 28760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:03:46.873618 28760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:03:46.882230 28760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:41149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:41321
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:41321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:46.883344 28760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:46.885066 28760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:46.896288 28770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:46.896319 28779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:46.896347 28771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:46.896761 28760 server_base.cc:1034] running on GCE node
I20260812 07:03:46.897720 28760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:46.899004 28760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:46.900167 28760 hybrid_clock.cc:648] HybridClock initialized: now 1786518226900120 us; error 36 us; skew 500 ppm
I20260812 07:03:46.900498 28760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:03:46.904860 28760 webserver.cc:460] Webserver started at http://127.11.192.254:43999/ using document root <none> and password file <none>
I20260812 07:03:46.905510 28760 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:46.905611 28760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:46.905920 28760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:46.908711 28760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "66174073acd7479db5d1a0858a71a476"
format_stamp: "Formatted at 2026-08-12 07:03:46 on dist-test-slave-lgr5"
server_key: "d58f80b09d2261c88d12e28efa048d24"
server_key_iv: "673940ee4cc6a62407bc6ef6776f406a"
server_key_version: "encryptionkey@0"
I20260812 07:03:46.909446 28760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "66174073acd7479db5d1a0858a71a476"
format_stamp: "Formatted at 2026-08-12 07:03:46 on dist-test-slave-lgr5"
server_key: "d58f80b09d2261c88d12e28efa048d24"
server_key_iv: "673940ee4cc6a62407bc6ef6776f406a"
server_key_version: "encryptionkey@0"
I20260812 07:03:46.914258 28760 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 07:03:46.917423 28793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:46.918618 28760 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:03:46.918774 28760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "66174073acd7479db5d1a0858a71a476"
format_stamp: "Formatted at 2026-08-12 07:03:46 on dist-test-slave-lgr5"
server_key: "d58f80b09d2261c88d12e28efa048d24"
server_key_iv: "673940ee4cc6a62407bc6ef6776f406a"
server_key_version: "encryptionkey@0"
I20260812 07:03:46.918958 28760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:46.950524 28760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:46.951359 28760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:46.951632 28760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:46.968535 28760 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:41321
I20260812 07:03:46.968556 28871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:41321 every 8 connection(s)
I20260812 07:03:46.969792 28760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:03:46.973431 28874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:46.975507 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 28760
I20260812 07:03:46.975718 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:03:46.976555 12035 external_mini_cluster.cc:1428] Setting key ffa5aa9ab7084be2a738c8a4d02ea70e
I20260812 07:03:46.985271 28874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476: Bootstrap starting.
I20260812 07:03:46.988624 28874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:46.989681 28874 log.cc:824] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476: Log is configured to *not* fsync() on all Append() calls
I20260812 07:03:46.989706 28880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:46.978627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59288 (local address 127.11.192.254:41321)
0812 07:03:46.979551 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:46.979562 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:46.979601 (+ 39us) server_negotiation.cc:407] Connection header received
0812 07:03:46.981250 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:46.981272 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:46.981697 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:46.982021 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:46.982623 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:46.983866 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:46.985645 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:46.986282 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:46.986403 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:46.987612 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:46.987619 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:46.987631 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:03:46.988163 (+ 532us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:46.988254 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:03:46.988275 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:46.988367 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:46.988563 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 07:03:46.988827 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":350,"threads_started":1}
I20260812 07:03:46.992434 28874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476: No bootstrap required, opened a new log
I20260812 07:03:46.999464 28874 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66174073acd7479db5d1a0858a71a476" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 41321 } }
I20260812 07:03:46.999805 28874 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:46.999889 28874 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66174073acd7479db5d1a0858a71a476, State: Initialized, Role: FOLLOWER
I20260812 07:03:47.000402 28874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66174073acd7479db5d1a0858a71a476" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 41321 } }
I20260812 07:03:47.000625 28874 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:03:47.000748 28874 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:03:47.000909 28874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:47.002880 28874 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66174073acd7479db5d1a0858a71a476" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 41321 } }
I20260812 07:03:47.003347 28874 leader_election.cc:302] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66174073acd7479db5d1a0858a71a476; no voters:
I20260812 07:03:47.003878 28874 leader_election.cc:288] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:03:47.003995 28885 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:03:47.004658 28885 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476 [term 1 LEADER]: Becoming Leader. State: Replica: 66174073acd7479db5d1a0858a71a476, State: Running, Role: LEADER
I20260812 07:03:47.005177 28885 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66174073acd7479db5d1a0858a71a476" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 41321 } }
I20260812 07:03:47.005772 28874 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:03:47.008008 28889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66174073acd7479db5d1a0858a71a476. Latest consensus state: current_term: 1 leader_uuid: "66174073acd7479db5d1a0858a71a476" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66174073acd7479db5d1a0858a71a476" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 41321 } } }
I20260812 07:03:47.008409 28889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:47.008893 28886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66174073acd7479db5d1a0858a71a476" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66174073acd7479db5d1a0858a71a476" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 41321 } } }
I20260812 07:03:47.009203 28886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:47.009925 28896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:03:47.014524 28896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:03:47.023178 28896 catalog_manager.cc:1347] Generated new cluster ID: 700ff173e96943a7a2e113afab228862
I20260812 07:03:47.023286 28896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:03:47.045275 28896 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:03:47.047060 28896 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:03:47.053905 28896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66174073acd7479db5d1a0858a71a476: Generated new TSK 0
I20260812 07:03:47.054838 28896 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:03:47.062868 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:41321
--builtin_ntp_servers=127.11.192.212:41149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 07:03:47.387959 28914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:47.388242 28914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:47.388469 28914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:47.395480 28914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:47.395567 28914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:47.395701 28914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:03:47.404572 28914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:41149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:41321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:47.405690 28914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:47.407407 28914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:47.419772 28922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:47.419732 28928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:47.419801 28930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:47.420261 28914 server_base.cc:1034] running on GCE node
I20260812 07:03:47.420928 28914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:47.421779 28914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:47.423038 28914 hybrid_clock.cc:648] HybridClock initialized: now 1786518227422980 us; error 48 us; skew 500 ppm
I20260812 07:03:47.423341 28914 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:03:47.425611 28914 webserver.cc:460] Webserver started at http://127.11.192.193:40575/ using document root <none> and password file <none>
I20260812 07:03:47.426250 28914 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:47.426360 28914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:47.426667 28914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:47.429404 28914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "0714b05294cf4ceabbe530d8ce5ef25f"
format_stamp: "Formatted at 2026-08-12 07:03:47 on dist-test-slave-lgr5"
server_key: "61b23a0f3fa5785e68317d8547b7dae3"
server_key_iv: "e1a7fd8cbb42bf7b0837066330f92936"
server_key_version: "encryptionkey@0"
I20260812 07:03:47.430143 28914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "0714b05294cf4ceabbe530d8ce5ef25f"
format_stamp: "Formatted at 2026-08-12 07:03:47 on dist-test-slave-lgr5"
server_key: "61b23a0f3fa5785e68317d8547b7dae3"
server_key_iv: "e1a7fd8cbb42bf7b0837066330f92936"
server_key_version: "encryptionkey@0"
I20260812 07:03:47.434845 28914 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:03:47.438050 28942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:47.439216 28914 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:03:47.439366 28914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "0714b05294cf4ceabbe530d8ce5ef25f"
format_stamp: "Formatted at 2026-08-12 07:03:47 on dist-test-slave-lgr5"
server_key: "61b23a0f3fa5785e68317d8547b7dae3"
server_key_iv: "e1a7fd8cbb42bf7b0837066330f92936"
server_key_version: "encryptionkey@0"
I20260812 07:03:47.439558 28914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:47.481346 28914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:47.482146 28914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:47.482432 28914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:47.483343 28914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:47.484752 28914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:47.484848 28914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:47.484954 28914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:47.485030 28914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:47.514626 28914 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:40825
I20260812 07:03:47.514676 29164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:40825 every 8 connection(s)
I20260812 07:03:47.515842 28914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:03:47.519881 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 28914
I20260812 07:03:47.520112 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:03:47.520747 12035 external_mini_cluster.cc:1428] Setting key 4b981025158f5274421b57af6d9df0c9
I20260812 07:03:47.523677 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:41321
--builtin_ntp_servers=127.11.192.212:41149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 07:03:47.528298 29168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:47.518346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:51255 (local address 127.11.192.254:41321)
0812 07:03:47.518738 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:47.518742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:47.520138 (+ 1396us) server_negotiation.cc:407] Connection header received
0812 07:03:47.521859 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:47.521865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:47.521948 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:47.522033 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:47.523168 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:47.524195 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:47.525691 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:47.526248 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:47.526365 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:47.527528 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:47.527534 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:47.527538 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:03:47.527559 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:47.527608 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:03:47.527623 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:47.527707 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:47.527874 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 07:03:47.527963 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":290,"thread_start_us":200,"threads_started":1}
I20260812 07:03:47.529199 29173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:47.518937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:41321 (local address 127.11.192.193:51255)
0812 07:03:47.519852 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:47.519896 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:47.521551 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:47.522281 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:47.522292 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:47.522405 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:47.522998 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:47.523018 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:47.524332 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:47.524336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:47.525550 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:47.525563 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:47.526425 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:47.526430 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:47.526632 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:47.527243 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:03:47.527245 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:47.527285 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:03:47.527355 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:47.527702 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:47.527708 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:47.527714 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 07:03:47.527847 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 07:03:47.528156 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":247,"threads_started":1}
I20260812 07:03:47.530550 29165 heartbeater.cc:347] Connected to a master server at 127.11.192.254:41321
I20260812 07:03:47.530807 29165 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:47.531512 29165 heartbeater.cc:511] Master 127.11.192.254:41321 requested a full tablet report, sending...
I20260812 07:03:47.533591 28818 ts_manager.cc:194] Registered new tserver with Master: 0714b05294cf4ceabbe530d8ce5ef25f (127.11.192.193:40825)
I20260812 07:03:47.535315 28818 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.192.193:51255
W20260812 07:03:47.846148 29174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:47.846391 29174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:47.846515 29174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:47.853416 29174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:47.853518 29174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:47.853662 29174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:03:47.862234 29174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:41149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:41321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:47.863368 29174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:47.865053 29174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:47.877310 29181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:47.877286 29182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:47.877360 29184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:47.878159 29174 server_base.cc:1034] running on GCE node
I20260812 07:03:47.878777 29174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:47.879653 29174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:47.880844 29174 hybrid_clock.cc:648] HybridClock initialized: now 1786518227880785 us; error 52 us; skew 500 ppm
I20260812 07:03:47.881143 29174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:03:47.883178 29174 webserver.cc:460] Webserver started at http://127.11.192.194:36511/ using document root <none> and password file <none>
I20260812 07:03:47.883805 29174 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:47.883905 29174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:47.884258 29174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:47.886926 29174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "6859c22d62ef4ef98332fa4ee37c18cc"
format_stamp: "Formatted at 2026-08-12 07:03:47 on dist-test-slave-lgr5"
server_key: "83ea5ea1829676583be0137c352eb7ec"
server_key_iv: "7c7be2a43cda6f3512bf0325201e1340"
server_key_version: "encryptionkey@0"
I20260812 07:03:47.887694 29174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "6859c22d62ef4ef98332fa4ee37c18cc"
format_stamp: "Formatted at 2026-08-12 07:03:47 on dist-test-slave-lgr5"
server_key: "83ea5ea1829676583be0137c352eb7ec"
server_key_iv: "7c7be2a43cda6f3512bf0325201e1340"
server_key_version: "encryptionkey@0"
I20260812 07:03:47.892529 29174 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260812 07:03:47.895717 29199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:47.896924 29174 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:03:47.897085 29174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "6859c22d62ef4ef98332fa4ee37c18cc"
format_stamp: "Formatted at 2026-08-12 07:03:47 on dist-test-slave-lgr5"
server_key: "83ea5ea1829676583be0137c352eb7ec"
server_key_iv: "7c7be2a43cda6f3512bf0325201e1340"
server_key_version: "encryptionkey@0"
I20260812 07:03:47.897281 29174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:47.919078 29174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:47.919808 29174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:47.920121 29174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:47.920976 29174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:47.922345 29174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:47.922442 29174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:47.922547 29174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:47.922617 29174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:47.951864 29174 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:43777
I20260812 07:03:47.951947 29424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:43777 every 8 connection(s)
I20260812 07:03:47.953063 29174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:03:47.959982 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 29174
I20260812 07:03:47.960170 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:03:47.960625 12035 external_mini_cluster.cc:1428] Setting key a9c0748ba8bc5c7211ca39561f049dc6
I20260812 07:03:47.963524 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:41321
--builtin_ntp_servers=127.11.192.212:41149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 07:03:47.964891 29168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:47.955316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:52575 (local address 127.11.192.254:41321)
0812 07:03:47.955495 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:47.955500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:47.956874 (+ 1374us) server_negotiation.cc:407] Connection header received
0812 07:03:47.958499 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:47.958503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:47.958569 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:47.958660 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:47.959565 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:47.960369 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:47.962219 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:47.962711 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:47.962805 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:47.964245 (+ 1440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:47.964250 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:47.964253 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:03:47.964270 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:47.964304 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:03:47.964315 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:47.964382 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:47.964592 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 07:03:47.964666 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 07:03:47.966087 29430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:47.955880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:41321 (local address 127.11.192.194:52575)
0812 07:03:47.956735 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:47.956769 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:47.958285 (+ 1516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:47.958828 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:47.958836 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:47.958920 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:47.959432 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:47.959448 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:47.960530 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:47.960535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:47.962078 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:47.962090 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:47.962925 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:47.962932 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:47.963213 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:47.963924 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:03:47.963927 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:47.963972 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:03:47.964063 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:47.964398 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:47.964408 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:47.964415 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 07:03:47.964589 (+ 174us) client_negotiation.cc:240] Negotiation successful
0812 07:03:47.964938 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":275,"threads_started":1}
I20260812 07:03:47.967288 29428 heartbeater.cc:347] Connected to a master server at 127.11.192.254:41321
I20260812 07:03:47.967578 29428 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:47.968400 29428 heartbeater.cc:511] Master 127.11.192.254:41321 requested a full tablet report, sending...
I20260812 07:03:47.969868 28818 ts_manager.cc:194] Registered new tserver with Master: 6859c22d62ef4ef98332fa4ee37c18cc (127.11.192.194:43777)
I20260812 07:03:47.970923 28818 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.192.194:52575
W20260812 07:03:48.294359 29434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:48.294680 29434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:48.294816 29434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:48.302330 29434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:48.302423 29434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:48.302558 29434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:03:48.311369 29434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:41149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:41321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:48.312575 29434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:48.314261 29434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:48.326597 29447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:48.326701 29443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:48.326707 29444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:48.327615 29434 server_base.cc:1034] running on GCE node
I20260812 07:03:48.328230 29434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:48.329061 29434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:48.330250 29434 hybrid_clock.cc:648] HybridClock initialized: now 1786518228330199 us; error 40 us; skew 500 ppm
I20260812 07:03:48.330569 29434 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:03:48.332576 29434 webserver.cc:460] Webserver started at http://127.11.192.195:45081/ using document root <none> and password file <none>
I20260812 07:03:48.333211 29434 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:48.333331 29434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:48.333630 29434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:48.336272 29434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "996059042d61480a91be0805e59e3893"
format_stamp: "Formatted at 2026-08-12 07:03:48 on dist-test-slave-lgr5"
server_key: "49c08ede60cbe42fcbf00a96f0e98318"
server_key_iv: "965d8f47acc560b96f96fabf61953fd8"
server_key_version: "encryptionkey@0"
I20260812 07:03:48.337005 29434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "996059042d61480a91be0805e59e3893"
format_stamp: "Formatted at 2026-08-12 07:03:48 on dist-test-slave-lgr5"
server_key: "49c08ede60cbe42fcbf00a96f0e98318"
server_key_iv: "965d8f47acc560b96f96fabf61953fd8"
server_key_version: "encryptionkey@0"
I20260812 07:03:48.341725 29434 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:03:48.345088 29454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:48.346271 29434 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260812 07:03:48.346427 29434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "996059042d61480a91be0805e59e3893"
format_stamp: "Formatted at 2026-08-12 07:03:48 on dist-test-slave-lgr5"
server_key: "49c08ede60cbe42fcbf00a96f0e98318"
server_key_iv: "965d8f47acc560b96f96fabf61953fd8"
server_key_version: "encryptionkey@0"
I20260812 07:03:48.346611 29434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:48.368656 29434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:48.369433 29434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:48.369731 29434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:48.370646 29434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:48.372100 29434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:48.372200 29434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:48.372310 29434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:48.372390 29434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:48.404167 29434 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:43225
I20260812 07:03:48.404249 29647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:43225 every 8 connection(s)
I20260812 07:03:48.405476 29434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:03:48.414659 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 29434
I20260812 07:03:48.414865 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:03:48.415323 12035 external_mini_cluster.cc:1428] Setting key 63eaa4f44ae1ce05e1da20bcdac3a932
I20260812 07:03:48.417675 29168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:48.407967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:47877 (local address 127.11.192.254:41321)
0812 07:03:48.408162 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:48.408167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:48.409519 (+ 1352us) server_negotiation.cc:407] Connection header received
0812 07:03:48.411221 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:48.411226 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:48.411307 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:48.411395 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:48.412480 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:48.413212 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:48.414742 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:48.415196 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:48.415309 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:48.416735 (+ 1426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:48.416742 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:48.416747 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:03:48.416771 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:48.416814 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:03:48.416833 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:48.416925 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:48.417173 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 07:03:48.417277 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:03:48.418569 29660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:48.408504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:41321 (local address 127.11.192.195:47877)
0812 07:03:48.409371 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:48.409405 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:48.410966 (+ 1561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:48.411649 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:48.411657 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:48.411736 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:48.412318 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:48.412337 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:48.413377 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:48.413381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:48.414603 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:48.414616 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:48.415411 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:48.415418 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:48.415704 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:48.416365 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:03:48.416369 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:48.416420 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:03:48.416522 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:48.416935 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:48.416948 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:48.416957 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 07:03:48.417145 (+ 188us) client_negotiation.cc:240] Negotiation successful
0812 07:03:48.417510 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":278,"threads_started":1}
I20260812 07:03:48.419827 29655 heartbeater.cc:347] Connected to a master server at 127.11.192.254:41321
I20260812 07:03:48.420110 29655 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:48.420795 29655 heartbeater.cc:511] Master 127.11.192.254:41321 requested a full tablet report, sending...
I20260812 07:03:48.422073 28818 ts_manager.cc:194] Registered new tserver with Master: 996059042d61480a91be0805e59e3893 (127.11.192.195:43225)
I20260812 07:03:48.422817 28818 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.192.195:47877
I20260812 07:03:48.430397 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 07:03:48.435966 29168 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:48.434608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59300 (local address 127.11.192.254:41321)
0812 07:03:48.434775 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:48.434780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:48.435239 (+ 459us) server_negotiation.cc:407] Connection header received
0812 07:03:48.435363 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:48.435367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:48.435433 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:48.435522 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:48.435800 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59300: BlockingRecv error: recv got EOF from 127.0.0.1:59300 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 07:03:48.436729 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 28914
I20260812 07:03:48.443547 29677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:48.437517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54822 (local address 127.11.192.193:40825)
0812 07:03:48.437977 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:48.437983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:48.438008 (+ 25us) server_negotiation.cc:407] Connection header received
0812 07:03:48.438060 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:48.438066 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:48.438241 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:48.438355 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:48.438642 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:48.439621 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:48.440834 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:48.441367 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:48.441460 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:48.442339 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:48.442346 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:48.442357 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:03:48.442891 (+ 534us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:48.442933 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:03:48.442952 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:48.443068 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:48.443152 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 07:03:48.443260 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":223,"threads_started":1}
I20260812 07:03:48.444638 29154 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:48.524840 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 28914
I20260812 07:03:48.547338 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 29174
I20260812 07:03:48.554853 29679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:48.548244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49774 (local address 127.11.192.194:43777)
0812 07:03:48.548759 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:48.548765 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:48.548793 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:03:48.548855 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:48.548861 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:48.549077 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:48.549206 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:48.549521 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:48.550594 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:48.551859 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:48.552560 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:48.552656 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:48.553506 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:48.553514 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:48.553527 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:03:48.554119 (+ 592us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:48.554166 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:03:48.554187 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:48.554272 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:48.554364 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 07:03:48.554515 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":261,"threads_started":1}
I20260812 07:03:48.556140 29410 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:48.635807 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 29174
I20260812 07:03:48.655078 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 29434
I20260812 07:03:48.662741 29683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:48.655988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57450 (local address 127.11.192.195:43225)
0812 07:03:48.656662 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:48.656669 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:48.656701 (+ 32us) server_negotiation.cc:407] Connection header received
0812 07:03:48.656770 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:48.656776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:48.657023 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:48.657149 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:48.657582 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:48.658588 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:48.659757 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:48.660355 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:48.660453 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:48.661413 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:48.661421 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:48.661435 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:03:48.662000 (+ 565us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:48.662048 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:03:48.662069 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:48.662152 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:48.662260 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 07:03:48.662369 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":360,"threads_started":1}
I20260812 07:03:48.663944 29638 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:48.743884 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 29434
I20260812 07:03:48.771801 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 28760
I20260812 07:03:48.773079 28865 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:48.873888 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 28760
2026-08-12T07:03:48Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2412 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T07:03:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:03:48Z Disabled control of system clock
I20260812 07:03:48.959177 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:35655
--webserver_interface=127.11.192.254
--webserver_port=0
--builtin_ntp_servers=127.11.192.212:42555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:35655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 07:03:49.277823 29692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:49.278066 29692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:49.278153 29692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:49.284915 29692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:03:49.285005 29692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:49.285053 29692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:49.285105 29692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:03:49.285147 29692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:03:49.293697 29692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:42555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:35655
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:35655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:49.294729 29692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:49.296388 29692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:49.307143 29708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:49.307173 29710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:49.307220 29715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:49.307904 29692 server_base.cc:1034] running on GCE node
I20260812 07:03:49.308593 29692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:49.309790 29692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:49.310994 29692 hybrid_clock.cc:648] HybridClock initialized: now 1786518229310950 us; error 40 us; skew 500 ppm
I20260812 07:03:49.311295 29692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:03:49.313336 29692 webserver.cc:460] Webserver started at http://127.11.192.254:46501/ using document root <none> and password file <none>
I20260812 07:03:49.313943 29692 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:49.314039 29692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:49.314337 29692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:49.317057 29692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "b694957284ae4a5eb60fa11e65fb35c0"
format_stamp: "Formatted at 2026-08-12 07:03:49 on dist-test-slave-lgr5"
server_key: "78dacef264186e4afbb5f8fcf7bf7ed6"
server_key_iv: "8fa7c660f1d3d3c6b2defc8c2dd7a77b"
server_key_version: "encryptionkey@0"
I20260812 07:03:49.317785 29692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "b694957284ae4a5eb60fa11e65fb35c0"
format_stamp: "Formatted at 2026-08-12 07:03:49 on dist-test-slave-lgr5"
server_key: "78dacef264186e4afbb5f8fcf7bf7ed6"
server_key_iv: "8fa7c660f1d3d3c6b2defc8c2dd7a77b"
server_key_version: "encryptionkey@0"
I20260812 07:03:49.322377 29692 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 07:03:49.325562 29723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:49.326717 29692 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:03:49.326867 29692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "b694957284ae4a5eb60fa11e65fb35c0"
format_stamp: "Formatted at 2026-08-12 07:03:49 on dist-test-slave-lgr5"
server_key: "78dacef264186e4afbb5f8fcf7bf7ed6"
server_key_iv: "8fa7c660f1d3d3c6b2defc8c2dd7a77b"
server_key_version: "encryptionkey@0"
I20260812 07:03:49.327047 29692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:49.332113 29692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:49.332777 29692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:49.333041 29692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:49.348799 29692 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:35655
I20260812 07:03:49.348835 29818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:35655 every 8 connection(s)
I20260812 07:03:49.349936 29692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:03:49.353260 29820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:49.357906 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 29692
I20260812 07:03:49.358076 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:03:49.358700 12035 external_mini_cluster.cc:1428] Setting key 52f0e4d84e324460d19fd2d6dd9554fc
I20260812 07:03:49.364315 29820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0: Bootstrap starting.
I20260812 07:03:49.367691 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:49.360536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58010 (local address 127.11.192.254:35655)
0812 07:03:49.361714 (+ 1178us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:49.361727 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:49.361785 (+ 58us) server_negotiation.cc:407] Connection header received
0812 07:03:49.363961 (+ 2176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:49.363989 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:49.364502 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:49.364899 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:49.365866 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:49.365875 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:49.365894 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:03:49.365912 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:49.365989 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:03:49.366011 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:49.366222 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:49.366340 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 07:03:49.366611 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":371,"threads_started":1}
I20260812 07:03:49.367991 29820 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:49.369081 29820 log.cc:824] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0: Log is configured to *not* fsync() on all Append() calls
I20260812 07:03:49.371686 29820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0: No bootstrap required, opened a new log
I20260812 07:03:49.378661 29820 raft_consensus.cc:348] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b694957284ae4a5eb60fa11e65fb35c0" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35655 } }
I20260812 07:03:49.378949 29820 raft_consensus.cc:374] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:49.379029 29820 raft_consensus.cc:729] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b694957284ae4a5eb60fa11e65fb35c0, State: Initialized, Role: FOLLOWER
I20260812 07:03:49.379518 29820 consensus_queue.cc:260] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b694957284ae4a5eb60fa11e65fb35c0" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35655 } }
I20260812 07:03:49.379714 29820 raft_consensus.cc:388] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:03:49.379804 29820 raft_consensus.cc:482] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:03:49.379940 29820 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:49.382211 29820 raft_consensus.cc:504] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b694957284ae4a5eb60fa11e65fb35c0" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35655 } }
I20260812 07:03:49.382609 29820 leader_election.cc:302] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b694957284ae4a5eb60fa11e65fb35c0; no voters:
I20260812 07:03:49.383074 29820 leader_election.cc:288] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:03:49.383176 29831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:03:49.383805 29831 raft_consensus.cc:686] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0 [term 1 LEADER]: Becoming Leader. State: Replica: b694957284ae4a5eb60fa11e65fb35c0, State: Running, Role: LEADER
I20260812 07:03:49.384351 29831 consensus_queue.cc:237] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b694957284ae4a5eb60fa11e65fb35c0" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35655 } }
I20260812 07:03:49.384917 29820 sys_catalog.cc:564] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:03:49.386960 29837 sys_catalog.cc:455] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b694957284ae4a5eb60fa11e65fb35c0. Latest consensus state: current_term: 1 leader_uuid: "b694957284ae4a5eb60fa11e65fb35c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b694957284ae4a5eb60fa11e65fb35c0" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35655 } } }
I20260812 07:03:49.386930 29836 sys_catalog.cc:455] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b694957284ae4a5eb60fa11e65fb35c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b694957284ae4a5eb60fa11e65fb35c0" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35655 } } }
I20260812 07:03:49.387326 29837 sys_catalog.cc:458] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:49.387321 29836 sys_catalog.cc:458] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:49.392642 29845 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:03:49.399176 29845 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:03:49.407650 29845 catalog_manager.cc:1347] Generated new cluster ID: 61d8997f143646cb9a8466154fe91fd3
I20260812 07:03:49.407738 29845 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:03:49.427937 29845 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:03:49.428323 29845 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:03:49.435629 29845 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b694957284ae4a5eb60fa11e65fb35c0: Generated new TSK 0
I20260812 07:03:49.436434 29845 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:03:49.451920 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:35655
--builtin_ntp_servers=127.11.192.212:42555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 07:03:49.774427 29873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:49.774691 29873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:49.774827 29873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:49.782132 29873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:49.782229 29873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:49.782382 29873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:03:49.791430 29873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:42555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:35655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:49.792575 29873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:49.794332 29873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:49.806401 29880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:49.806473 29881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:49.806444 29885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:49.807396 29873 server_base.cc:1034] running on GCE node
I20260812 07:03:49.807982 29873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:49.808894 29873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:49.810103 29873 hybrid_clock.cc:648] HybridClock initialized: now 1786518229810057 us; error 30 us; skew 500 ppm
I20260812 07:03:49.810410 29873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:03:49.812558 29873 webserver.cc:460] Webserver started at http://127.11.192.193:36515/ using document root <none> and password file <none>
I20260812 07:03:49.813199 29873 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:49.813316 29873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:49.813651 29873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:49.816484 29873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "a0d028e20afe48008335e9fb161fa7ca"
format_stamp: "Formatted at 2026-08-12 07:03:49 on dist-test-slave-lgr5"
server_key: "4bcabc96ecc2a3dfbc0b5e0ac04f120a"
server_key_iv: "84269c19ca4683563412e7ddfb640769"
server_key_version: "encryptionkey@0"
I20260812 07:03:49.817320 29873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "a0d028e20afe48008335e9fb161fa7ca"
format_stamp: "Formatted at 2026-08-12 07:03:49 on dist-test-slave-lgr5"
server_key: "4bcabc96ecc2a3dfbc0b5e0ac04f120a"
server_key_iv: "84269c19ca4683563412e7ddfb640769"
server_key_version: "encryptionkey@0"
I20260812 07:03:49.822374 29873 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:03:49.825744 29894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:49.826946 29873 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:03:49.827114 29873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "a0d028e20afe48008335e9fb161fa7ca"
format_stamp: "Formatted at 2026-08-12 07:03:49 on dist-test-slave-lgr5"
server_key: "4bcabc96ecc2a3dfbc0b5e0ac04f120a"
server_key_iv: "84269c19ca4683563412e7ddfb640769"
server_key_version: "encryptionkey@0"
I20260812 07:03:49.827312 29873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:49.832711 29873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:49.833417 29873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:49.833654 29873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:49.834481 29873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:49.835759 29873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:49.835853 29873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:49.835956 29873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:49.836033 29873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:49.864202 29873 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:37637
I20260812 07:03:49.864261 30099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:37637 every 8 connection(s)
I20260812 07:03:49.865409 29873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:03:49.868399 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 29873
I20260812 07:03:49.868558 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:03:49.869196 12035 external_mini_cluster.cc:1428] Setting key 61e096bcc6e889f596217420ea653820
I20260812 07:03:49.871984 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:35655
--builtin_ntp_servers=127.11.192.212:42555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 07:03:49.873261 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:49.867810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:43461 (local address 127.11.192.254:35655)
0812 07:03:49.868014 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:49.868020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:49.869471 (+ 1451us) server_negotiation.cc:407] Connection header received
0812 07:03:49.871225 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:49.871230 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:49.871292 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:49.871402 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:49.872633 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:49.872638 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:49.872642 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:03:49.872661 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:49.872701 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:03:49.872716 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:49.872837 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:49.872957 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 07:03:49.873017 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:03:49.874044 30106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:49.868321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:35655 (local address 127.11.192.193:43461)
0812 07:03:49.869313 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:49.869357 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:49.870985 (+ 1628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:49.871571 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:49.871578 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:49.871660 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:49.872341 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:03:49.872343 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:49.872379 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:03:49.872460 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:49.872799 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:49.872804 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:49.872810 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 07:03:49.873002 (+ 192us) client_negotiation.cc:240] Negotiation successful
0812 07:03:49.873182 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":294,"threads_started":1}
I20260812 07:03:49.875351 30100 heartbeater.cc:347] Connected to a master server at 127.11.192.254:35655
I20260812 07:03:49.875625 30100 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:49.876186 30100 heartbeater.cc:511] Master 127.11.192.254:35655 requested a full tablet report, sending...
I20260812 07:03:49.878064 29752 ts_manager.cc:194] Registered new tserver with Master: a0d028e20afe48008335e9fb161fa7ca (127.11.192.193:37637)
W20260812 07:03:50.208606 30111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:50.208894 30111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:50.209045 30111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:50.216248 30111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:50.216358 30111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:50.216524 30111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:03:50.225616 30111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:42555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:35655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:50.226857 30111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:50.228735 30111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:50.241758 30125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:50.241861 30121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:50.241832 30123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:50.242559 30111 server_base.cc:1034] running on GCE node
I20260812 07:03:50.243203 30111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:50.244186 30111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:50.245395 30111 hybrid_clock.cc:648] HybridClock initialized: now 1786518230245341 us; error 41 us; skew 500 ppm
I20260812 07:03:50.245704 30111 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:03:50.247855 30111 webserver.cc:460] Webserver started at http://127.11.192.194:44127/ using document root <none> and password file <none>
I20260812 07:03:50.248575 30111 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:50.248689 30111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:50.249025 30111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:50.251870 30111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "44a80852444742668ad41db5cc5e8a2b"
format_stamp: "Formatted at 2026-08-12 07:03:50 on dist-test-slave-lgr5"
server_key: "2428c87efc400212c0f228268550625a"
server_key_iv: "141f796bcda6cf43b355b91104b16fc0"
server_key_version: "encryptionkey@0"
I20260812 07:03:50.252743 30111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "44a80852444742668ad41db5cc5e8a2b"
format_stamp: "Formatted at 2026-08-12 07:03:50 on dist-test-slave-lgr5"
server_key: "2428c87efc400212c0f228268550625a"
server_key_iv: "141f796bcda6cf43b355b91104b16fc0"
server_key_version: "encryptionkey@0"
I20260812 07:03:50.257961 30111 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:03:50.261394 30140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:50.262650 30111 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:03:50.262828 30111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "44a80852444742668ad41db5cc5e8a2b"
format_stamp: "Formatted at 2026-08-12 07:03:50 on dist-test-slave-lgr5"
server_key: "2428c87efc400212c0f228268550625a"
server_key_iv: "141f796bcda6cf43b355b91104b16fc0"
server_key_version: "encryptionkey@0"
I20260812 07:03:50.263041 30111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:50.268777 30111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:50.269574 30111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:50.269851 30111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:50.270743 30111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:50.272186 30111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:50.272292 30111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:50.272410 30111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:50.272490 30111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:50.304015 30111 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:40529
I20260812 07:03:50.304100 30348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:40529 every 8 connection(s)
I20260812 07:03:50.305369 30111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:03:50.309583 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 30111
I20260812 07:03:50.309751 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:03:50.310307 12035 external_mini_cluster.cc:1428] Setting key 0e02e254d66a2838ead8020caf7a4870
I20260812 07:03:50.313480 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:35655
--builtin_ntp_servers=127.11.192.212:42555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 07:03:50.313737 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:50.307766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:54919 (local address 127.11.192.254:35655)
0812 07:03:50.307918 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:50.307923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:50.309325 (+ 1402us) server_negotiation.cc:407] Connection header received
0812 07:03:50.311408 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:50.311415 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:50.311474 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:50.311569 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:50.312902 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:50.312908 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:50.312913 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:03:50.312935 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:50.312976 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:03:50.312994 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:50.313140 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:50.313262 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 07:03:50.313330 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 07:03:50.314430 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:50.308280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:35655 (local address 127.11.192.194:54919)
0812 07:03:50.309147 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:50.309195 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:50.311165 (+ 1970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:50.311810 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:50.311821 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:50.311929 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:50.312615 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:03:50.312618 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:50.312653 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:03:50.312734 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:50.313088 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:50.313094 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:50.313098 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 07:03:50.313291 (+ 193us) client_negotiation.cc:240] Negotiation successful
0812 07:03:50.313471 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":268,"threads_started":1}
I20260812 07:03:50.315788 30350 heartbeater.cc:347] Connected to a master server at 127.11.192.254:35655
I20260812 07:03:50.316048 30350 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:50.316608 30350 heartbeater.cc:511] Master 127.11.192.254:35655 requested a full tablet report, sending...
I20260812 07:03:50.317878 29752 ts_manager.cc:194] Registered new tserver with Master: 44a80852444742668ad41db5cc5e8a2b (127.11.192.194:40529)
W20260812 07:03:50.637827 30353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:50.638058 30353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:50.638162 30353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:50.644963 30353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:50.645047 30353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:50.645186 30353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:03:50.653709 30353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:42555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:35655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:50.654850 30353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:50.656509 30353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:50.668497 30361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:50.668530 30366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:50.668563 30362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:50.669518 30353 server_base.cc:1034] running on GCE node
I20260812 07:03:50.670006 30353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:50.670816 30353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:50.671988 30353 hybrid_clock.cc:648] HybridClock initialized: now 1786518230671942 us; error 46 us; skew 500 ppm
I20260812 07:03:50.672281 30353 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:03:50.674381 30353 webserver.cc:460] Webserver started at http://127.11.192.195:35529/ using document root <none> and password file <none>
I20260812 07:03:50.674983 30353 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:50.675086 30353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:50.675392 30353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:50.678048 30353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "7af96e1fb85a494688ad19b373fc2352"
format_stamp: "Formatted at 2026-08-12 07:03:50 on dist-test-slave-lgr5"
server_key: "4aa9308fc6a930b234129a91d7630439"
server_key_iv: "5ec29bd1d803119fd78ef77aba275745"
server_key_version: "encryptionkey@0"
I20260812 07:03:50.678828 30353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "7af96e1fb85a494688ad19b373fc2352"
format_stamp: "Formatted at 2026-08-12 07:03:50 on dist-test-slave-lgr5"
server_key: "4aa9308fc6a930b234129a91d7630439"
server_key_iv: "5ec29bd1d803119fd78ef77aba275745"
server_key_version: "encryptionkey@0"
I20260812 07:03:50.683566 30353 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 07:03:50.686861 30376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:50.688140 30353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 07:03:50.688295 30353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "7af96e1fb85a494688ad19b373fc2352"
format_stamp: "Formatted at 2026-08-12 07:03:50 on dist-test-slave-lgr5"
server_key: "4aa9308fc6a930b234129a91d7630439"
server_key_iv: "5ec29bd1d803119fd78ef77aba275745"
server_key_version: "encryptionkey@0"
I20260812 07:03:50.688526 30353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:50.693849 30353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:50.694574 30353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:50.694800 30353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:50.695652 30353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:50.697022 30353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:50.697115 30353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:50.697216 30353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:50.697283 30353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:50.727181 30353 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:35317
I20260812 07:03:50.727231 30573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:35317 every 8 connection(s)
I20260812 07:03:50.728435 30353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:03:50.729256 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 30353
I20260812 07:03:50.729408 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:03:50.729929 12035 external_mini_cluster.cc:1428] Setting key 60831aa5ec831a981e38b0bbfd492e13
I20260812 07:03:50.736253 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:50.731031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:52867 (local address 127.11.192.254:35655)
0812 07:03:50.731257 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:50.731263 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:50.733053 (+ 1790us) server_negotiation.cc:407] Connection header received
0812 07:03:50.734474 (+ 1421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:50.734478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:50.734529 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:50.734599 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:50.735702 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:50.735706 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:50.735709 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:03:50.735723 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:50.735754 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:03:50.735765 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:50.735848 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:50.735972 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 07:03:50.736029 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 07:03:50.737046 30578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:50.731737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:35655 (local address 127.11.192.195:52867)
0812 07:03:50.732754 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:50.732796 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:50.734245 (+ 1449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:50.734776 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:50.734784 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:50.734878 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:50.735439 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:03:50.735441 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:50.735484 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:03:50.735561 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:50.735834 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:50.735840 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:50.735844 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 07:03:50.736028 (+ 184us) client_negotiation.cc:240] Negotiation successful
0812 07:03:50.736243 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":327,"threads_started":1}
I20260812 07:03:50.738155 30574 heartbeater.cc:347] Connected to a master server at 127.11.192.254:35655
I20260812 07:03:50.738422 30574 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:50.738946 30574 heartbeater.cc:511] Master 127.11.192.254:35655 requested a full tablet report, sending...
I20260812 07:03:50.740170 29752 ts_manager.cc:194] Registered new tserver with Master: 7af96e1fb85a494688ad19b373fc2352 (127.11.192.195:35317)
I20260812 07:03:50.744905 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 07:03:50.750291 29829 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:03:50.749030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57896 (local address 127.11.192.254:35655)
0812 07:03:50.749161 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:50.749165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:50.749656 (+ 491us) server_negotiation.cc:407] Connection header received
0812 07:03:50.749780 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:50.749783 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:50.749829 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:50.749896 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:50.750117 (+ 221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57896: BlockingRecv error: recv got EOF from 127.0.0.1:57896 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 07:03:50.750880 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 29873
I20260812 07:03:50.753876 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:50.751717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54726 (local address 127.11.192.193:37637)
0812 07:03:50.752219 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:50.752225 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:50.752248 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:03:50.752302 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:50.752307 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:50.752468 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:50.752590 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:50.753341 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:50.753347 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:50.753357 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:03:50.753375 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:50.753415 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:03:50.753429 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:50.753539 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:50.753586 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 07:03:50.753659 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":225,"threads_started":1}
I20260812 07:03:50.754930 30093 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:50.833076 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 29873
I20260812 07:03:50.856310 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 30111
I20260812 07:03:50.859578 30594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:50.857211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56038 (local address 127.11.192.194:40529)
0812 07:03:50.857722 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:50.857729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:50.857756 (+ 27us) server_negotiation.cc:407] Connection header received
0812 07:03:50.857820 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:50.857827 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:50.858044 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:50.858166 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:50.858974 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:50.858981 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:50.858992 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:03:50.859010 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:50.859050 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:03:50.859064 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:50.859164 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:50.859236 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 07:03:50.859321 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":228,"threads_started":1}
I20260812 07:03:50.860817 30335 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:50.939208 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 30111
I20260812 07:03:50.955359 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 30353
I20260812 07:03:50.959311 30597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:50.956388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38716 (local address 127.11.192.195:35317)
0812 07:03:50.957018 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:50.957025 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:50.957053 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:03:50.957114 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:50.957120 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:50.957340 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:50.957485 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:50.958603 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:50.958609 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:50.958622 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:03:50.958639 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:50.958678 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:03:50.958691 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:50.958845 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:50.958926 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 07:03:50.959030 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":310,"threads_started":1}
I20260812 07:03:50.960549 30565 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:51.040407 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 30353
I20260812 07:03:51.064724 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 29692
I20260812 07:03:51.065912 29809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:51.168416 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 29692
2026-08-12T07:03:51Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2289 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:03:51 dist-test-slave-lgr5 krb5kdc[30607](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:03:51 dist-test-slave-lgr5 krb5kdc[30607](info): set up 2 sockets
Aug 12 07:03:51 dist-test-slave-lgr5 krb5kdc[30607](info): commencing operation
krb5kdc: starting...
W20260812 07:03:52.424264 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:03:52 dist-test-slave-lgr5 krb5kdc[30607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:03:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:03:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:03:52.650996 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:40601
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:35491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:40601
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:52.963891 30634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:52.964190 30634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:52.964286 30634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:52.971513 30634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:03:52.971606 30634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:52.971657 30634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:52.971714 30634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:03:52.971756 30634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:03:52.981021 30634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:35491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:40601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:40601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:52.982139 30634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:52.983304 30634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:52.994166 30642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:52.994215 30640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:52.994234 30647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:52.994837 30634 server_base.cc:1034] running on GCE node
I20260812 07:03:52.995499 30634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:52.996742 30634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:52.997936 30634 hybrid_clock.cc:648] HybridClock initialized: now 1786518232997893 us; error 29 us; skew 500 ppm
I20260812 07:03:52.998256 30634 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:53 dist-test-slave-lgr5 krb5kdc[30607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518233, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:53.004132 30634 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:03:53.005465 30634 webserver.cc:460] Webserver started at http://127.11.192.254:42375/ using document root <none> and password file <none>
I20260812 07:03:53.006103 30634 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:53.006220 30634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:53.006513 30634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:53.009430 30634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "ae4395d919784583bb016d4c404cedb5"
format_stamp: "Formatted at 2026-08-12 07:03:53 on dist-test-slave-lgr5"
server_key: "43c581f495a044b21222a456a97ea0bb"
server_key_iv: "465da07ca5cf62b82a8dd82f86b7155c"
server_key_version: "encryptionkey@0"
I20260812 07:03:53.010171 30634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "ae4395d919784583bb016d4c404cedb5"
format_stamp: "Formatted at 2026-08-12 07:03:53 on dist-test-slave-lgr5"
server_key: "43c581f495a044b21222a456a97ea0bb"
server_key_iv: "465da07ca5cf62b82a8dd82f86b7155c"
server_key_version: "encryptionkey@0"
I20260812 07:03:53.015094 30634 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:03:53.018322 30658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:53.019552 30634 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 07:03:53.019713 30634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "ae4395d919784583bb016d4c404cedb5"
format_stamp: "Formatted at 2026-08-12 07:03:53 on dist-test-slave-lgr5"
server_key: "43c581f495a044b21222a456a97ea0bb"
server_key_iv: "465da07ca5cf62b82a8dd82f86b7155c"
server_key_version: "encryptionkey@0"
I20260812 07:03:53.019876 30634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:53.030100 30634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:53.034713 30634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:53.034960 30634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:53.050359 30634 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:40601
I20260812 07:03:53.050379 30762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:40601 every 8 connection(s)
I20260812 07:03:53.051625 30634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:03:53.055078 30763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:53.059367 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 30634
I20260812 07:03:53.059553 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:03:53.060217 12035 external_mini_cluster.cc:1428] Setting key 69efabdebf8a6e9838088e7c83548a91
I20260812 07:03:53.065272 30763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5: Bootstrap starting.
I20260812 07:03:53.068670 30763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:53.069850 30763 log.cc:824] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5: Log is configured to *not* fsync() on all Append() calls
I20260812 07:03:53.072506 30763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5: No bootstrap required, opened a new log
Aug 12 07:03:53 dist-test-slave-lgr5 krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:03:53.079864 30763 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4395d919784583bb016d4c404cedb5" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40601 } }
I20260812 07:03:53.080241 30763 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:53.080344 30763 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae4395d919784583bb016d4c404cedb5, State: Initialized, Role: FOLLOWER
I20260812 07:03:53.080926 30763 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4395d919784583bb016d4c404cedb5" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40601 } }
I20260812 07:03:53.081173 30763 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:03:53.081303 30763 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:03:53.081470 30763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:53.081614 30769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:53.062031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36608 (local address 127.11.192.254:40601)
0812 07:03:53.062811 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:53.062821 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:53.062867 (+ 46us) server_negotiation.cc:407] Connection header received
0812 07:03:53.064510 (+ 1643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:53.064532 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:53.064863 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:53.065170 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:53.066994 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:53.068295 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:53.070058 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:53.070748 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:53.070930 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:53.074679 (+ 3749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:53.074686 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:53.074703 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:53.074740 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:53.077721 (+ 2981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:53.078363 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:53.078369 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:53.078375 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:53.078561 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:53.078924 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:53.078927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:53.078930 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:53.079421 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:53.079706 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:53.080150 (+ 444us) server_negotiation.cc:299] Negotiation successful
0812 07:03:53.080579 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":364,"threads_started":1}
I20260812 07:03:53.083683 30763 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4395d919784583bb016d4c404cedb5" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40601 } }
I20260812 07:03:53.084189 30763 leader_election.cc:302] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae4395d919784583bb016d4c404cedb5; no voters:
I20260812 07:03:53.084718 30763 leader_election.cc:288] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:03:53.084823 30772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:03:53.085450 30772 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5 [term 1 LEADER]: Becoming Leader. State: Replica: ae4395d919784583bb016d4c404cedb5, State: Running, Role: LEADER
I20260812 07:03:53.085983 30772 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4395d919784583bb016d4c404cedb5" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40601 } }
I20260812 07:03:53.086494 30763 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:03:53.088614 30773 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae4395d919784583bb016d4c404cedb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4395d919784583bb016d4c404cedb5" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40601 } } }
I20260812 07:03:53.088785 30774 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae4395d919784583bb016d4c404cedb5. Latest consensus state: current_term: 1 leader_uuid: "ae4395d919784583bb016d4c404cedb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4395d919784583bb016d4c404cedb5" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40601 } } }
I20260812 07:03:53.089231 30773 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:53.089301 30774 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5 [sys.catalog]: This master's current role is: LEADER
I20260812 07:03:53.093015 30784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:03:53.098176 30784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:03:53.106601 30784 catalog_manager.cc:1347] Generated new cluster ID: 107af6b3e0c645449894926018123fc3
I20260812 07:03:53.106693 30784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:03:53.126899 30784 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:03:53.128160 30784 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:03:53.144639 30784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5: Generated new TSK 0
I20260812 07:03:53.145427 30784 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
I20260812 07:03:53.204385 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:40601
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:35491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:53.527664 30804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:53.527920 30804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:53.528029 30804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:53.534946 30804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:53.535039 30804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:53.535167 30804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:03:53.544257 30804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:35491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:40601
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:53.545413 30804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:53.546626 30804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:53.558967 30811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:53.558941 30813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:53.558900 30818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:53.559653 30804 server_base.cc:1034] running on GCE node
I20260812 07:03:53.560227 30804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:53.561054 30804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:53.562245 30804 hybrid_clock.cc:648] HybridClock initialized: now 1786518233562207 us; error 26 us; skew 500 ppm
I20260812 07:03:53.562521 30804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:53 dist-test-slave-lgr5 krb5kdc[30607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518233, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:53.565475 30804 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:03:53.566746 30804 webserver.cc:460] Webserver started at http://127.11.192.193:36053/ using document root <none> and password file <none>
I20260812 07:03:53.567365 30804 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:53.567481 30804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:53.567759 30804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:53.570552 30804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "ed9afa23e4564cf8971ef064c9894833"
format_stamp: "Formatted at 2026-08-12 07:03:53 on dist-test-slave-lgr5"
server_key: "d022796d716c7e1a7ed913dba42468b5"
server_key_iv: "994ba4317511733abf4f6dab8e70289b"
server_key_version: "encryptionkey@0"
I20260812 07:03:53.571365 30804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "ed9afa23e4564cf8971ef064c9894833"
format_stamp: "Formatted at 2026-08-12 07:03:53 on dist-test-slave-lgr5"
server_key: "d022796d716c7e1a7ed913dba42468b5"
server_key_iv: "994ba4317511733abf4f6dab8e70289b"
server_key_version: "encryptionkey@0"
I20260812 07:03:53.576184 30804 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260812 07:03:53.579336 30833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:53.580580 30804 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:03:53.580732 30804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "ed9afa23e4564cf8971ef064c9894833"
format_stamp: "Formatted at 2026-08-12 07:03:53 on dist-test-slave-lgr5"
server_key: "d022796d716c7e1a7ed913dba42468b5"
server_key_iv: "994ba4317511733abf4f6dab8e70289b"
server_key_version: "encryptionkey@0"
I20260812 07:03:53.580906 30804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:53.598593 30804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:53.602098 30804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:53.602344 30804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:53.603179 30804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:53.604439 30804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:53.604517 30804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:53.604640 30804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:53.604693 30804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:53.633909 30804 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:44741
I20260812 07:03:53.633963 31041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:44741 every 8 connection(s)
I20260812 07:03:53.635082 30804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:03:53.635694 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 30804
I20260812 07:03:53.635882 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:03:53.636711 12035 external_mini_cluster.cc:1428] Setting key fa0853475b46543054f339f18e0e429f
Aug 12 07:03:53 dist-test-slave-lgr5 krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518233, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:03:53.654424 31050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:53.637799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:41259 (local address 127.11.192.254:40601)
0812 07:03:53.638255 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:53.638260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:53.639228 (+ 968us) server_negotiation.cc:407] Connection header received
0812 07:03:53.641021 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:53.641026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:53.641098 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:53.641199 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:53.642536 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:53.643448 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:53.645116 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:53.645642 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:53.645762 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:53.649059 (+ 3297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:53.649065 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:53.649069 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:53.649105 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:53.651537 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:53.652236 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:53.652241 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:53.652245 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:53.652404 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:53.652837 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:53.652842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:53.652844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:53.653176 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:53.653314 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:53.653897 (+ 583us) server_negotiation.cc:299] Negotiation successful
0812 07:03:53.654111 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":225,"threads_started":1}
I20260812 07:03:53.655079 31051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:53.638241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:40601 (local address 127.11.192.193:41259)
0812 07:03:53.639059 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:53.639104 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:53.640766 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:53.641385 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:53.641393 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:53.641865 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:53.642373 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:53.642390 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:53.643590 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:53.643595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:53.644968 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:53.644981 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:53.645821 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:53.645825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:53.646008 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:53.646649 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:53.646681 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:53.648837 (+ 2156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:53.651657 (+ 2820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:53.651663 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:53.651673 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:53.652105 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:53.652519 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:53.652522 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:53.652525 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:53.652721 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:53.653277 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:53.653282 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:53.653557 (+ 275us) client_negotiation.cc:769] Sending connection context
0812 07:03:53.653760 (+ 203us) client_negotiation.cc:240] Negotiation successful
0812 07:03:53.654112 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":339,"threads_started":1}
I20260812 07:03:53.656458 31042 heartbeater.cc:347] Connected to a master server at 127.11.192.254:40601
I20260812 07:03:53.656723 31042 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:53.657390 31042 heartbeater.cc:511] Master 127.11.192.254:40601 requested a full tablet report, sending...
I20260812 07:03:53.659456 30695 ts_manager.cc:194] Registered new tserver with Master: ed9afa23e4564cf8971ef064c9894833 (127.11.192.193:44741)
I20260812 07:03:53.661378 30695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:41259
WARNING: no policy specified for kudu/127.11.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
I20260812 07:03:53.693229 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:40601
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:35491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:54.008008 31058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:54.008324 31058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:54.008466 31058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:54.015471 31058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:54.015580 31058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:54.015733 31058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:03:54.024991 31058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:35491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:40601
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:54.026099 31058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:54.027304 31058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:54.040565 31069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:54.040608 31067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:54.040921 31066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:54.041487 31058 server_base.cc:1034] running on GCE node
I20260812 07:03:54.042026 31058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:54.042840 31058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:54.044030 31058 hybrid_clock.cc:648] HybridClock initialized: now 1786518234043980 us; error 38 us; skew 500 ppm
I20260812 07:03:54.044358 31058 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:54 dist-test-slave-lgr5 krb5kdc[30607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518234, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:54.047251 31058 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:03:54.048664 31058 webserver.cc:460] Webserver started at http://127.11.192.194:34041/ using document root <none> and password file <none>
I20260812 07:03:54.049283 31058 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:54.049407 31058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:54.049686 31058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:54.052452 31058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "33f083bf00114bfd8f54341ebca205bf"
format_stamp: "Formatted at 2026-08-12 07:03:54 on dist-test-slave-lgr5"
server_key: "dd4dbfb61a1dd3d7a7082cf44d56dfe9"
server_key_iv: "44772742a5cf51cba817f993c620f47f"
server_key_version: "encryptionkey@0"
I20260812 07:03:54.053184 31058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "33f083bf00114bfd8f54341ebca205bf"
format_stamp: "Formatted at 2026-08-12 07:03:54 on dist-test-slave-lgr5"
server_key: "dd4dbfb61a1dd3d7a7082cf44d56dfe9"
server_key_iv: "44772742a5cf51cba817f993c620f47f"
server_key_version: "encryptionkey@0"
I20260812 07:03:54.058048 31058 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:03:54.061337 31082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:54.062666 31058 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:03:54.062816 31058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "33f083bf00114bfd8f54341ebca205bf"
format_stamp: "Formatted at 2026-08-12 07:03:54 on dist-test-slave-lgr5"
server_key: "dd4dbfb61a1dd3d7a7082cf44d56dfe9"
server_key_iv: "44772742a5cf51cba817f993c620f47f"
server_key_version: "encryptionkey@0"
I20260812 07:03:54.062994 31058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:54.074271 31058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:54.077735 31058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:54.077996 31058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:54.078882 31058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:54.080163 31058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:54.080237 31058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:54.080348 31058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:54.080397 31058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:54.108644 31058 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:42581
I20260812 07:03:54.108695 31287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:42581 every 8 connection(s)
I20260812 07:03:54.109839 31058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:03:54.113029 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 31058
I20260812 07:03:54.113202 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:03:54.113775 12035 external_mini_cluster.cc:1428] Setting key f767959c3037f9fd8d2206de677cf5c3
Aug 12 07:03:54 dist-test-slave-lgr5 krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518234, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:03:54.130326 31050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:54.112107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:33661 (local address 127.11.192.254:40601)
0812 07:03:54.112322 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:54.112329 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:54.113629 (+ 1300us) server_negotiation.cc:407] Connection header received
0812 07:03:54.115686 (+ 2057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:54.115692 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:54.115774 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:54.115874 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:54.117767 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.118487 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.120438 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.120841 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.120932 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:54.124722 (+ 3790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:54.124727 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:54.124731 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:54.124763 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:54.127198 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:54.127973 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:54.127979 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:54.127982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:54.128171 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:54.128548 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:54.128553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:54.128556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:54.128878 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:54.129033 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:54.129767 (+ 734us) server_negotiation.cc:299] Negotiation successful
0812 07:03:54.130000 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:03:54.131016 31293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:54.112580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:40601 (local address 127.11.192.194:33661)
0812 07:03:54.113450 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:54.113515 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:54.115402 (+ 1887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:54.116155 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:54.116166 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:54.116849 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:54.117584 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:54.117608 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.118629 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.118634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:54.120312 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:54.120332 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.121042 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.121049 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:54.121299 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:54.121958 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:54.121984 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:54.124512 (+ 2528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:54.127327 (+ 2815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:54.127335 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:54.127347 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:54.127849 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:54.128265 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:54.128269 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:54.128271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:54.128443 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:54.129038 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:54.129044 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:54.129375 (+ 331us) client_negotiation.cc:769] Sending connection context
0812 07:03:54.129616 (+ 241us) client_negotiation.cc:240] Negotiation successful
0812 07:03:54.129984 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":337,"threads_started":1}
I20260812 07:03:54.132357 31290 heartbeater.cc:347] Connected to a master server at 127.11.192.254:40601
I20260812 07:03:54.132654 31290 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:54.133361 31290 heartbeater.cc:511] Master 127.11.192.254:40601 requested a full tablet report, sending...
I20260812 07:03:54.134793 30695 ts_manager.cc:194] Registered new tserver with Master: 33f083bf00114bfd8f54341ebca205bf (127.11.192.194:42581)
I20260812 07:03:54.135573 30695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:33661
WARNING: no policy specified for kudu/127.11.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
I20260812 07:03:54.172801 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:40601
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:35491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:03:54.494458 31297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:03:54.494740 31297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:03:54.494884 31297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:03:54.501827 31297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:03:54.501941 31297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:03:54.502095 31297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:03:54.511215 31297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:35491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:40601
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:03:54.512389 31297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:03:54.513633 31297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:03:54.525743 31310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:54.525701 31316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:03:54.525779 31307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:03:54.526571 31297 server_base.cc:1034] running on GCE node
I20260812 07:03:54.527102 31297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:03:54.527945 31297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:03:54.529134 31297 hybrid_clock.cc:648] HybridClock initialized: now 1786518234529093 us; error 39 us; skew 500 ppm
I20260812 07:03:54.529439 31297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:03:54 dist-test-slave-lgr5 krb5kdc[30607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518234, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:03:54.532298 31297 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:03:54.533648 31297 webserver.cc:460] Webserver started at http://127.11.192.195:45563/ using document root <none> and password file <none>
I20260812 07:03:54.534252 31297 fs_manager.cc:359] Metadata directory not provided
I20260812 07:03:54.534375 31297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:03:54.534652 31297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:03:54.537405 31297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "24d13a1e85a24167bd6923b9d68c5f31"
format_stamp: "Formatted at 2026-08-12 07:03:54 on dist-test-slave-lgr5"
server_key: "1a3c76c4d1d089797b0f90b1656ee21c"
server_key_iv: "9c32f7ef3fd4d3e54a46e873d325dea4"
server_key_version: "encryptionkey@0"
I20260812 07:03:54.538149 31297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "24d13a1e85a24167bd6923b9d68c5f31"
format_stamp: "Formatted at 2026-08-12 07:03:54 on dist-test-slave-lgr5"
server_key: "1a3c76c4d1d089797b0f90b1656ee21c"
server_key_iv: "9c32f7ef3fd4d3e54a46e873d325dea4"
server_key_version: "encryptionkey@0"
I20260812 07:03:54.542908 31297 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:03:54.546070 31334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:54.547410 31297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:03:54.547576 31297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "24d13a1e85a24167bd6923b9d68c5f31"
format_stamp: "Formatted at 2026-08-12 07:03:54 on dist-test-slave-lgr5"
server_key: "1a3c76c4d1d089797b0f90b1656ee21c"
server_key_iv: "9c32f7ef3fd4d3e54a46e873d325dea4"
server_key_version: "encryptionkey@0"
I20260812 07:03:54.547749 31297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:03:54.566704 31297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:03:54.570609 31297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:03:54.570899 31297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:03:54.571859 31297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:03:54.573415 31297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:03:54.573505 31297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:54.573644 31297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:03:54.573709 31297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:03:54.605928 31297 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:44679
I20260812 07:03:54.605960 31548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:44679 every 8 connection(s)
I20260812 07:03:54.607184 31297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:03:54.614818 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 31297
I20260812 07:03:54.614995 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:03:54.615521 12035 external_mini_cluster.cc:1428] Setting key 30165ceefbfaa3535125ba9b4f44c836
Aug 12 07:03:54 dist-test-slave-lgr5 krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518234, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:03:54.625443 31050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:54.609620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:36335 (local address 127.11.192.254:40601)
0812 07:03:54.609811 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:54.609815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:54.610849 (+ 1034us) server_negotiation.cc:407] Connection header received
0812 07:03:54.612578 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:54.612583 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:54.612648 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:54.612734 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:54.614152 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.615199 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.616835 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.617413 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.617526 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:54.620783 (+ 3257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:54.620788 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:54.620791 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:54.620821 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:54.622993 (+ 2172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:54.623604 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:54.623608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:54.623610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:54.623723 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:54.624058 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:54.624061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:54.624063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:54.624382 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:54.624520 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:54.625036 (+ 516us) server_negotiation.cc:299] Negotiation successful
0812 07:03:54.625202 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 07:03:54.626235 31553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:54.610012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:40601 (local address 127.11.192.195:36335)
0812 07:03:54.610698 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:54.610734 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:54.612342 (+ 1608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:54.612915 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:54.612922 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:54.613424 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:03:54.614006 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:54.614024 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.615331 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.615335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:54.616706 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:54.616719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.617640 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.617648 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:54.617921 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:54.618599 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:03:54.618626 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:03:54.620610 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:03:54.623118 (+ 2508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:54.623125 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:54.623134 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:54.623509 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:54.623797 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:03:54.623800 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:03:54.623802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:03:54.623968 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:03:54.624486 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:03:54.624493 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:03:54.624760 (+ 267us) client_negotiation.cc:769] Sending connection context
0812 07:03:54.624961 (+ 201us) client_negotiation.cc:240] Negotiation successful
0812 07:03:54.625303 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":261,"threads_started":1}
I20260812 07:03:54.627388 31550 heartbeater.cc:347] Connected to a master server at 127.11.192.254:40601
I20260812 07:03:54.627640 31550 heartbeater.cc:464] Registering TS with master...
I20260812 07:03:54.628337 31550 heartbeater.cc:511] Master 127.11.192.254:40601 requested a full tablet report, sending...
I20260812 07:03:54.629665 30694 ts_manager.cc:194] Registered new tserver with Master: 24d13a1e85a24167bd6923b9d68c5f31 (127.11.192.195:44679)
I20260812 07:03:54.630441 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:03:54.630739 30694 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:36335
I20260812 07:03:54.646749 31050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:54.635114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36614 (local address 127.11.192.254:40601)
0812 07:03:54.635264 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:54.635269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:54.635818 (+ 549us) server_negotiation.cc:407] Connection header received
0812 07:03:54.635973 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:54.635977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:54.636032 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:54.636155 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:54.637850 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.638546 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.639984 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.640413 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.640546 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:54.642497 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:54.642501 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:54.642504 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:54.642533 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:54.644715 (+ 2182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:54.645256 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:54.645259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:54.645261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:54.645375 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:54.645669 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:54.645672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:54.645674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:54.645925 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:54.646098 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:54.646324 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 07:03:54.646470 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 07:03:54.650367 30694 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:03:54.656939 30694 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:03:54.664072 31042 heartbeater.cc:503] Master 127.11.192.254:40601 was elected leader, sending a full tablet report...
I20260812 07:03:54.679009 31573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:54.668352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40008 (local address 127.11.192.195:44679)
0812 07:03:54.670894 (+ 2542us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:54.670904 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:54.670939 (+ 35us) server_negotiation.cc:407] Connection header received
0812 07:03:54.671038 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:54.671046 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:54.671290 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:54.671498 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:54.672164 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.674181 (+ 2017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.676523 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.677865 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.678058 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:54.678225 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:54.678502 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 07:03:54.678669 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2365,"thread_start_us":2191,"threads_started":1}
I20260812 07:03:54.678994 31572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:54.668031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:44679 (local address 127.0.0.1:40008)
0812 07:03:54.668721 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:54.668779 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:54.668957 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:54.671534 (+ 2577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:54.671543 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:54.671577 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:54.671928 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:54.671939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.674302 (+ 2363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.674306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:54.676313 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:54.676328 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.678052 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.678059 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:54.678245 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:54.678296 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 07:03:54.678430 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 07:03:54.678632 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":282,"threads_started":1}
I20260812 07:03:54.679427 31577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:54.668868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:42581 (local address 127.0.0.1:43682)
0812 07:03:54.669597 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:54.669614 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:54.669724 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:54.673073 (+ 3349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:54.673078 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:54.673105 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:54.673566 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:54.673579 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.675319 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.675325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:54.677110 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:54.677125 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.678600 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.678606 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:54.678769 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:54.678802 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:03:54.678870 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 07:03:54.679000 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"mutex_wait_us":4,"thread_start_us":454,"threads_started":1}
I20260812 07:03:54.679759 31581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:54.669180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:44741 (local address 127.0.0.1:54276)
0812 07:03:54.670161 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:54.670179 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:54.670286 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:54.672715 (+ 2429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:54.672720 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:54.672743 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:54.673109 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:54.673120 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.675007 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.675013 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:54.677199 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:54.677215 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.678930 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.678937 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:54.679105 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:54.679150 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 07:03:54.679213 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 07:03:54.679325 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"mutex_wait_us":42,"thread_start_us":836,"threads_started":1}
I20260812 07:03:54.680307 31583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:54.669813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54276 (local address 127.11.192.193:44741)
0812 07:03:54.672248 (+ 2435us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:54.672256 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:54.672280 (+ 24us) server_negotiation.cc:407] Connection header received
0812 07:03:54.672364 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:54.672371 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:54.672546 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:54.672703 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:54.673323 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.674867 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.677379 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.678694 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.679584 (+ 890us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:54.679753 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:54.679820 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 07:03:54.679974 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2233,"thread_start_us":2067,"threads_started":1}
I20260812 07:03:54.680501 31582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:54.669775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43682 (local address 127.11.192.194:42581)
0812 07:03:54.672481 (+ 2706us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:54.672489 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:54.672514 (+ 25us) server_negotiation.cc:407] Connection header received
0812 07:03:54.672605 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:54.672611 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:54.672814 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:54.673066 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:54.673776 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.675188 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.677302 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.678382 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.679791 (+ 1409us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:54.679937 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:54.680009 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 07:03:54.680207 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2549,"thread_start_us":2432,"threads_started":1}
I20260812 07:03:54.683184 31422 tablet_service.cc:1467] Processing CreateTablet for tablet 6e5fd1d9c47b461799d200a9b22d372a (DEFAULT_TABLE table=test-table [id=6c9dba1449874840b8736c3489bec89f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:03:54.683359 31165 tablet_service.cc:1467] Processing CreateTablet for tablet 6e5fd1d9c47b461799d200a9b22d372a (DEFAULT_TABLE table=test-table [id=6c9dba1449874840b8736c3489bec89f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:03:54.683703 30927 tablet_service.cc:1467] Processing CreateTablet for tablet 6e5fd1d9c47b461799d200a9b22d372a (DEFAULT_TABLE table=test-table [id=6c9dba1449874840b8736c3489bec89f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:03:54.684649 31165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e5fd1d9c47b461799d200a9b22d372a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:54.684808 31422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e5fd1d9c47b461799d200a9b22d372a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:54.684964 30927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e5fd1d9c47b461799d200a9b22d372a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:03:54.695791 31587 tablet_bootstrap.cc:492] T 6e5fd1d9c47b461799d200a9b22d372a P 33f083bf00114bfd8f54341ebca205bf: Bootstrap starting.
I20260812 07:03:54.698539 31589 tablet_bootstrap.cc:492] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833: Bootstrap starting.
I20260812 07:03:54.699314 31592 tablet_bootstrap.cc:492] T 6e5fd1d9c47b461799d200a9b22d372a P 24d13a1e85a24167bd6923b9d68c5f31: Bootstrap starting.
I20260812 07:03:54.699867 31587 tablet_bootstrap.cc:654] T 6e5fd1d9c47b461799d200a9b22d372a P 33f083bf00114bfd8f54341ebca205bf: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:54.701184 31587 log.cc:824] T 6e5fd1d9c47b461799d200a9b22d372a P 33f083bf00114bfd8f54341ebca205bf: Log is configured to *not* fsync() on all Append() calls
I20260812 07:03:54.702518 31589 tablet_bootstrap.cc:654] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:54.702796 31592 tablet_bootstrap.cc:654] T 6e5fd1d9c47b461799d200a9b22d372a P 24d13a1e85a24167bd6923b9d68c5f31: Neither blocks nor log segments found. Creating new log.
I20260812 07:03:54.703686 31592 log.cc:824] T 6e5fd1d9c47b461799d200a9b22d372a P 24d13a1e85a24167bd6923b9d68c5f31: Log is configured to *not* fsync() on all Append() calls
I20260812 07:03:54.703707 31589 log.cc:824] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833: Log is configured to *not* fsync() on all Append() calls
I20260812 07:03:54.704319 31587 tablet_bootstrap.cc:492] T 6e5fd1d9c47b461799d200a9b22d372a P 33f083bf00114bfd8f54341ebca205bf: No bootstrap required, opened a new log
I20260812 07:03:54.704632 31587 ts_tablet_manager.cc:1397] T 6e5fd1d9c47b461799d200a9b22d372a P 33f083bf00114bfd8f54341ebca205bf: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260812 07:03:54.706224 31589 tablet_bootstrap.cc:492] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833: No bootstrap required, opened a new log
I20260812 07:03:54.706452 31589 ts_tablet_manager.cc:1397] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 07:03:54.706521 31592 tablet_bootstrap.cc:492] T 6e5fd1d9c47b461799d200a9b22d372a P 24d13a1e85a24167bd6923b9d68c5f31: No bootstrap required, opened a new log
I20260812 07:03:54.706772 31592 ts_tablet_manager.cc:1397] T 6e5fd1d9c47b461799d200a9b22d372a P 24d13a1e85a24167bd6923b9d68c5f31: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 07:03:54.713572 31589 raft_consensus.cc:348] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d13a1e85a24167bd6923b9d68c5f31" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 44679 } } peers { permanent_uuid: "ed9afa23e4564cf8971ef064c9894833" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 44741 } } peers { permanent_uuid: "33f083bf00114bfd8f54341ebca205bf" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 42581 } }
I20260812 07:03:54.713950 31589 raft_consensus.cc:374] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:54.714056 31589 raft_consensus.cc:729] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed9afa23e4564cf8971ef064c9894833, State: Initialized, Role: FOLLOWER
I20260812 07:03:54.714610 31589 consensus_queue.cc:260] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d13a1e85a24167bd6923b9d68c5f31" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 44679 } } peers { permanent_uuid: "ed9afa23e4564cf8971ef064c9894833" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 44741 } } peers { permanent_uuid: "33f083bf00114bfd8f54341ebca205bf" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 42581 } }
I20260812 07:03:54.714556 31587 raft_consensus.cc:348] T 6e5fd1d9c47b461799d200a9b22d372a P 33f083bf00114bfd8f54341ebca205bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d13a1e85a24167bd6923b9d68c5f31" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 44679 } } peers { permanent_uuid: "ed9afa23e4564cf8971ef064c9894833" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 44741 } } peers { permanent_uuid: "33f083bf00114bfd8f54341ebca205bf" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 42581 } }
I20260812 07:03:54.715004 31587 raft_consensus.cc:374] T 6e5fd1d9c47b461799d200a9b22d372a P 33f083bf00114bfd8f54341ebca205bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:54.715157 31587 raft_consensus.cc:729] T 6e5fd1d9c47b461799d200a9b22d372a P 33f083bf00114bfd8f54341ebca205bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33f083bf00114bfd8f54341ebca205bf, State: Initialized, Role: FOLLOWER
I20260812 07:03:54.715875 31587 consensus_queue.cc:260] T 6e5fd1d9c47b461799d200a9b22d372a P 33f083bf00114bfd8f54341ebca205bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d13a1e85a24167bd6923b9d68c5f31" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 44679 } } peers { permanent_uuid: "ed9afa23e4564cf8971ef064c9894833" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 44741 } } peers { permanent_uuid: "33f083bf00114bfd8f54341ebca205bf" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 42581 } }
I20260812 07:03:54.716344 31589 ts_tablet_manager.cc:1428] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260812 07:03:54.716706 31592 raft_consensus.cc:348] T 6e5fd1d9c47b461799d200a9b22d372a P 24d13a1e85a24167bd6923b9d68c5f31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d13a1e85a24167bd6923b9d68c5f31" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 44679 } } peers { permanent_uuid: "ed9afa23e4564cf8971ef064c9894833" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 44741 } } peers { permanent_uuid: "33f083bf00114bfd8f54341ebca205bf" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 42581 } }
I20260812 07:03:54.717155 31592 raft_consensus.cc:374] T 6e5fd1d9c47b461799d200a9b22d372a P 24d13a1e85a24167bd6923b9d68c5f31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:03:54.717262 31592 raft_consensus.cc:729] T 6e5fd1d9c47b461799d200a9b22d372a P 24d13a1e85a24167bd6923b9d68c5f31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d13a1e85a24167bd6923b9d68c5f31, State: Initialized, Role: FOLLOWER
I20260812 07:03:54.717844 31592 consensus_queue.cc:260] T 6e5fd1d9c47b461799d200a9b22d372a P 24d13a1e85a24167bd6923b9d68c5f31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d13a1e85a24167bd6923b9d68c5f31" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 44679 } } peers { permanent_uuid: "ed9afa23e4564cf8971ef064c9894833" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 44741 } } peers { permanent_uuid: "33f083bf00114bfd8f54341ebca205bf" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 42581 } }
I20260812 07:03:54.723141 31290 heartbeater.cc:503] Master 127.11.192.254:40601 was elected leader, sending a full tablet report...
I20260812 07:03:54.724172 31587 ts_tablet_manager.cc:1428] T 6e5fd1d9c47b461799d200a9b22d372a P 33f083bf00114bfd8f54341ebca205bf: Time spent starting tablet: real 0.019s user 0.018s sys 0.001s
I20260812 07:03:54.725078 31550 heartbeater.cc:503] Master 127.11.192.254:40601 was elected leader, sending a full tablet report...
I20260812 07:03:54.725570 31592 ts_tablet_manager.cc:1428] T 6e5fd1d9c47b461799d200a9b22d372a P 24d13a1e85a24167bd6923b9d68c5f31: Time spent starting tablet: real 0.019s user 0.014s sys 0.005s
I20260812 07:03:54.727815 31599 raft_consensus.cc:482] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:03:54.728024 31599 raft_consensus.cc:504] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d13a1e85a24167bd6923b9d68c5f31" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 44679 } } peers { permanent_uuid: "ed9afa23e4564cf8971ef064c9894833" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 44741 } } peers { permanent_uuid: "33f083bf00114bfd8f54341ebca205bf" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 42581 } }
I20260812 07:03:54.729764 31599 leader_election.cc:288] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24d13a1e85a24167bd6923b9d68c5f31 (127.11.192.195:44679), 33f083bf00114bfd8f54341ebca205bf (127.11.192.194:42581)
I20260812 07:03:54.736054 31573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:54.730815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:34927 (local address 127.11.192.195:44679)
0812 07:03:54.731103 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:54.731107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:54.731120 (+ 13us) server_negotiation.cc:407] Connection header received
0812 07:03:54.731167 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:54.731171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:54.731219 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:54.731293 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:54.731843 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.732662 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.734426 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.735192 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.735335 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:54.735410 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:54.735700 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 07:03:54.735818 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183}
I20260812 07:03:54.736138 31605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:54.730146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:44679 (local address 127.11.192.193:34927)
0812 07:03:54.730780 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:54.730796 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:54.730921 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:54.731329 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:54.731333 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:54.731364 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:54.731708 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:54.731717 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.732783 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.732788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:54.734278 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:54.734292 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.735401 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.735407 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:54.735554 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:54.735578 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 07:03:54.735670 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 07:03:54.735796 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":372,"threads_started":1}
I20260812 07:03:54.737089 31460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e5fd1d9c47b461799d200a9b22d372a" candidate_uuid: "ed9afa23e4564cf8971ef064c9894833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d13a1e85a24167bd6923b9d68c5f31" is_pre_election: true
I20260812 07:03:54.737607 31606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:54.730230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:42581 (local address 127.11.192.193:43331)
0812 07:03:54.732606 (+ 2376us) negotiation.cc:107] Waiting for socket to connect
0812 07:03:54.732620 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:03:54.732736 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:03:54.733112 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:03:54.733115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:03:54.733139 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:03:54.733480 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:54.733489 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.734436 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.734441 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:54.735719 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:03:54.735731 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.737036 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.737041 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:03:54.737153 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:54.737176 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 07:03:54.737247 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 07:03:54.737326 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2260,"thread_start_us":2022,"threads_started":1}
I20260812 07:03:54.737654 31582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:54.730773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:43331 (local address 127.11.192.194:42581)
0812 07:03:54.730961 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:54.730966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:54.732699 (+ 1733us) server_negotiation.cc:407] Connection header received
0812 07:03:54.732836 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:54.732841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:54.732895 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:54.732966 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:03:54.733640 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.734309 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.735863 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.736839 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.736993 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:54.737057 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:54.737294 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 07:03:54.737413 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:03:54.737572 31460 raft_consensus.cc:2454] T 6e5fd1d9c47b461799d200a9b22d372a P 24d13a1e85a24167bd6923b9d68c5f31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed9afa23e4564cf8971ef064c9894833 in term 0.
I20260812 07:03:54.738202 30837 leader_election.cc:302] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24d13a1e85a24167bd6923b9d68c5f31, ed9afa23e4564cf8971ef064c9894833; no voters:
I20260812 07:03:54.738384 31205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e5fd1d9c47b461799d200a9b22d372a" candidate_uuid: "ed9afa23e4564cf8971ef064c9894833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33f083bf00114bfd8f54341ebca205bf" is_pre_election: true
I20260812 07:03:54.738535 31599 raft_consensus.cc:2781] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:03:54.738638 31599 raft_consensus.cc:482] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:03:54.738729 31599 raft_consensus.cc:3037] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:54.738818 31205 raft_consensus.cc:2454] T 6e5fd1d9c47b461799d200a9b22d372a P 33f083bf00114bfd8f54341ebca205bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed9afa23e4564cf8971ef064c9894833 in term 0.
I20260812 07:03:54.741073 31599 raft_consensus.cc:504] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d13a1e85a24167bd6923b9d68c5f31" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 44679 } } peers { permanent_uuid: "ed9afa23e4564cf8971ef064c9894833" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 44741 } } peers { permanent_uuid: "33f083bf00114bfd8f54341ebca205bf" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 42581 } }
I20260812 07:03:54.741645 31599 leader_election.cc:288] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833 [CANDIDATE]: Term 1 election: Requested vote from peers 24d13a1e85a24167bd6923b9d68c5f31 (127.11.192.195:44679), 33f083bf00114bfd8f54341ebca205bf (127.11.192.194:42581)
I20260812 07:03:54.741986 31460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e5fd1d9c47b461799d200a9b22d372a" candidate_uuid: "ed9afa23e4564cf8971ef064c9894833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d13a1e85a24167bd6923b9d68c5f31"
I20260812 07:03:54.742075 31205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e5fd1d9c47b461799d200a9b22d372a" candidate_uuid: "ed9afa23e4564cf8971ef064c9894833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33f083bf00114bfd8f54341ebca205bf"
I20260812 07:03:54.742152 31460 raft_consensus.cc:3037] T 6e5fd1d9c47b461799d200a9b22d372a P 24d13a1e85a24167bd6923b9d68c5f31 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:54.742245 31205 raft_consensus.cc:3037] T 6e5fd1d9c47b461799d200a9b22d372a P 33f083bf00114bfd8f54341ebca205bf [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:03:54.744256 31460 raft_consensus.cc:2454] T 6e5fd1d9c47b461799d200a9b22d372a P 24d13a1e85a24167bd6923b9d68c5f31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed9afa23e4564cf8971ef064c9894833 in term 1.
I20260812 07:03:54.744446 31205 raft_consensus.cc:2454] T 6e5fd1d9c47b461799d200a9b22d372a P 33f083bf00114bfd8f54341ebca205bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed9afa23e4564cf8971ef064c9894833 in term 1.
I20260812 07:03:54.744586 30837 leader_election.cc:302] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24d13a1e85a24167bd6923b9d68c5f31, ed9afa23e4564cf8971ef064c9894833; no voters:
I20260812 07:03:54.744868 31599 raft_consensus.cc:2781] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:03:54.745249 31599 raft_consensus.cc:686] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833 [term 1 LEADER]: Becoming Leader. State: Replica: ed9afa23e4564cf8971ef064c9894833, State: Running, Role: LEADER
I20260812 07:03:54.745654 31599 consensus_queue.cc:237] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d13a1e85a24167bd6923b9d68c5f31" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 44679 } } peers { permanent_uuid: "ed9afa23e4564cf8971ef064c9894833" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 44741 } } peers { permanent_uuid: "33f083bf00114bfd8f54341ebca205bf" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 42581 } }
I20260812 07:03:54.750088 30695 catalog_manager.cc:5495] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833 reported cstate change: term changed from 0 to 1, leader changed from <none> to ed9afa23e4564cf8971ef064c9894833 (127.11.192.193). New cstate: current_term: 1 leader_uuid: "ed9afa23e4564cf8971ef064c9894833" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24d13a1e85a24167bd6923b9d68c5f31" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed9afa23e4564cf8971ef064c9894833" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 44741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33f083bf00114bfd8f54341ebca205bf" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 42581 } health_report { overall_health: UNKNOWN } } }
I20260812 07:03:54.777093 31583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:54.771946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54280 (local address 127.11.192.193:44741)
0812 07:03:54.772141 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:54.772147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:54.772273 (+ 126us) server_negotiation.cc:407] Connection header received
0812 07:03:54.772481 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:54.772485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:54.772571 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:54.772687 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:03:54.773055 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.773743 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.775056 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.775448 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.775554 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:54.776039 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:03:54.776536 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:03:54.776666 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:54.776767 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 07:03:54.776845 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 07:03:54.784554 31205 raft_consensus.cc:1261] T 6e5fd1d9c47b461799d200a9b22d372a P 33f083bf00114bfd8f54341ebca205bf [term 1 FOLLOWER]: Refusing update from remote peer ed9afa23e4564cf8971ef064c9894833: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:03:54.784576 31460 raft_consensus.cc:1261] T 6e5fd1d9c47b461799d200a9b22d372a P 24d13a1e85a24167bd6923b9d68c5f31 [term 1 FOLLOWER]: Refusing update from remote peer ed9afa23e4564cf8971ef064c9894833: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:03:54.785549 31607 consensus_queue.cc:1035] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24d13a1e85a24167bd6923b9d68c5f31" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:03:54.785918 31599 consensus_queue.cc:1035] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33f083bf00114bfd8f54341ebca205bf" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 42581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:03:54.796411 31611 mvcc.cc:205] Tried to move back new op lower bound from 7317578689667461120 to 7317578689521840128. Current Snapshot: MvccSnapshot[applied={T|T < 7317578689667461120}]
I20260812 07:03:54.809971 30695 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:03:54.810266 30695 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:03:54.814659 30695 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae4395d919784583bb016d4c404cedb5: Sending DeleteTablet for 3 replicas of tablet 6e5fd1d9c47b461799d200a9b22d372a
I20260812 07:03:54.815821 31165 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e5fd1d9c47b461799d200a9b22d372a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:03:54 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:43682
I20260812 07:03:54.815943 31422 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e5fd1d9c47b461799d200a9b22d372a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:03:54 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:40008
I20260812 07:03:54.816363 30927 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e5fd1d9c47b461799d200a9b22d372a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:03:54 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:54276
I20260812 07:03:54.816462 31420 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e5fd1d9c47b461799d200a9b22d372a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:03:54 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:40008
I20260812 07:03:54.818387 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 30804
I20260812 07:03:54.828900 31632 tablet_replica.cc:330] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833: stopping tablet replica
I20260812 07:03:54.829309 31632 raft_consensus.cc:2229] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:03:54.829913 31632 raft_consensus.cc:2258] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:03:54.831714 31628 tablet_replica.cc:330] T 6e5fd1d9c47b461799d200a9b22d372a P 33f083bf00114bfd8f54341ebca205bf: stopping tablet replica
I20260812 07:03:54.831792 31632 ts_tablet_manager.cc:1905] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:03:54.833820 31628 raft_consensus.cc:2229] T 6e5fd1d9c47b461799d200a9b22d372a P 33f083bf00114bfd8f54341ebca205bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:03:54.838124 31628 raft_consensus.cc:2258] T 6e5fd1d9c47b461799d200a9b22d372a P 33f083bf00114bfd8f54341ebca205bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:03:54.839504 31632 ts_tablet_manager.cc:1918] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:03:54.839757 31632 log.cc:1196] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-0/wal/wals/6e5fd1d9c47b461799d200a9b22d372a
I20260812 07:03:54.840404 31632 ts_tablet_manager.cc:1939] T 6e5fd1d9c47b461799d200a9b22d372a P ed9afa23e4564cf8971ef064c9894833: Deleting consensus metadata
I20260812 07:03:54.841341 31628 ts_tablet_manager.cc:1905] T 6e5fd1d9c47b461799d200a9b22d372a P 33f083bf00114bfd8f54341ebca205bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:03:54.842165 30663 catalog_manager.cc:4841] TS ed9afa23e4564cf8971ef064c9894833 (127.11.192.193:44741): tablet 6e5fd1d9c47b461799d200a9b22d372a (table test-table [id=6c9dba1449874840b8736c3489bec89f]) successfully deleted
Aug 12 07:03:54 dist-test-slave-lgr5 krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.193@KRBTEST.COM
I20260812 07:03:54.848551 31631 tablet_replica.cc:330] T 6e5fd1d9c47b461799d200a9b22d372a P 24d13a1e85a24167bd6923b9d68c5f31: stopping tablet replica
I20260812 07:03:54.848901 31631 raft_consensus.cc:2229] T 6e5fd1d9c47b461799d200a9b22d372a P 24d13a1e85a24167bd6923b9d68c5f31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:03:54.849313 31631 raft_consensus.cc:2258] T 6e5fd1d9c47b461799d200a9b22d372a P 24d13a1e85a24167bd6923b9d68c5f31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:03:54.852545 31628 ts_tablet_manager.cc:1918] T 6e5fd1d9c47b461799d200a9b22d372a P 33f083bf00114bfd8f54341ebca205bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:03:54.852682 31628 log.cc:1196] T 6e5fd1d9c47b461799d200a9b22d372a P 33f083bf00114bfd8f54341ebca205bf: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/6e5fd1d9c47b461799d200a9b22d372a
I20260812 07:03:54.853137 31628 ts_tablet_manager.cc:1939] T 6e5fd1d9c47b461799d200a9b22d372a P 33f083bf00114bfd8f54341ebca205bf: Deleting consensus metadata
I20260812 07:03:54.854183 31583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:54.820921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54286 (local address 127.11.192.193:44741)
0812 07:03:54.821099 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:54.821105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:54.831076 (+ 9971us) server_negotiation.cc:407] Connection header received
0812 07:03:54.831274 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:54.831279 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:54.831357 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:54.831455 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:54.834099 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.835446 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.838588 (+ 3142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.839343 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.839443 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:54.846828 (+ 7385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:54.846844 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:54.846859 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:54.846899 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:54.850870 (+ 3971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:54.851760 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:54.851768 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:54.851772 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:54.851945 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:54.852476 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:54.852481 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:54.852485 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:54.852899 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:54.853206 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:54.853548 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 07:03:54.853781 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:03:54.854722 31631 ts_tablet_manager.cc:1905] T 6e5fd1d9c47b461799d200a9b22d372a P 24d13a1e85a24167bd6923b9d68c5f31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:03:54.855629 31035 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:54.860935 31631 ts_tablet_manager.cc:1918] T 6e5fd1d9c47b461799d200a9b22d372a P 24d13a1e85a24167bd6923b9d68c5f31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:03:54.861074 31631 log.cc:1196] T 6e5fd1d9c47b461799d200a9b22d372a P 24d13a1e85a24167bd6923b9d68c5f31: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/6e5fd1d9c47b461799d200a9b22d372a
I20260812 07:03:54.861474 31631 ts_tablet_manager.cc:1939] T 6e5fd1d9c47b461799d200a9b22d372a P 24d13a1e85a24167bd6923b9d68c5f31: Deleting consensus metadata
I20260812 07:03:54.862809 30663 catalog_manager.cc:4841] TS 24d13a1e85a24167bd6923b9d68c5f31 (127.11.192.195:44679): tablet 6e5fd1d9c47b461799d200a9b22d372a (table test-table [id=6c9dba1449874840b8736c3489bec89f]) successfully deleted
W20260812 07:03:54.863281 30663 catalog_manager.cc:4808] TS 24d13a1e85a24167bd6923b9d68c5f31 (127.11.192.195:44679): delete failed for tablet 6e5fd1d9c47b461799d200a9b22d372a because the tablet was not found. No further retry: Not found: Tablet not found: 6e5fd1d9c47b461799d200a9b22d372a
I20260812 07:03:54.864199 30665 catalog_manager.cc:4841] TS 33f083bf00114bfd8f54341ebca205bf (127.11.192.194:42581): tablet 6e5fd1d9c47b461799d200a9b22d372a (table test-table [id=6c9dba1449874840b8736c3489bec89f]) successfully deleted
I20260812 07:03:54.952900 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 30804
I20260812 07:03:54.980870 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 31058
Aug 12 07:03:54 dist-test-slave-lgr5 krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.194@KRBTEST.COM
I20260812 07:03:54.994706 31582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:54.981579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43698 (local address 127.11.192.194:42581)
0812 07:03:54.981760 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:54.981766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:54.981836 (+ 70us) server_negotiation.cc:407] Connection header received
0812 07:03:54.982028 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:54.982032 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:54.982107 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:54.982194 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:54.983390 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.984168 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.985394 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:54.985789 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:54.985901 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:54.989847 (+ 3946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:54.989861 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:54.989876 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:54.989910 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:54.992330 (+ 2420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:54.992959 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:54.992966 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:54.992969 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:54.993087 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:54.993392 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:54.993396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:54.993398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:54.993769 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:54.994008 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:54.994285 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 07:03:54.994448 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 07:03:54.995359 31279 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:55.082739 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 31058
I20260812 07:03:55.107951 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 31297
Aug 12 07:03:55 dist-test-slave-lgr5 krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.195@KRBTEST.COM
I20260812 07:03:55.122540 31573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:03:55.108727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40018 (local address 127.11.192.195:44679)
0812 07:03:55.108903 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 07:03:55.108909 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:03:55.108930 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:03:55.109098 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:03:55.109104 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:03:55.109184 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:03:55.109272 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:03:55.110521 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:55.111241 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:55.112518 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:03:55.112921 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:03:55.113046 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:03:55.117245 (+ 4199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:03:55.117260 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:03:55.117277 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:03:55.117309 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:03:55.119769 (+ 2460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:55.120443 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:55.120452 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:55.120454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:55.120591 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:03:55.120948 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:03:55.120952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:03:55.120954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:03:55.121413 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:03:55.121693 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0812 07:03:55.122033 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 07:03:55.122239 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:03:55.123178 31541 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:55.209650 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 31297
I20260812 07:03:55.230031 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 30634
I20260812 07:03:55.231115 30755 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:03:55.337517 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 30634
2026-08-12T07:03:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4169 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:03:55 dist-test-slave-lgr5 krb5kdc[31651](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:03:55 dist-test-slave-lgr5 krb5kdc[31651](info): set up 2 sockets
Aug 12 07:03:55 dist-test-slave-lgr5 krb5kdc[31651](info): commencing operation
krb5kdc: starting...
W20260812 07:03:56.587078 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:03:56 dist-test-slave-lgr5 krb5kdc[31651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:03:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:03:56Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.192.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.192.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.192.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/rangeradmin_127.11.192.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.192.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/rangeradmin_127.11.192.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.192.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.192.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.192.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/rangerlookup_127.11.192.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.192.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/rangerlookup_127.11.192.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.192.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.192.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.192.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/HTTP_127.11.192.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.192.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/HTTP_127.11.192.212@KRBTEST.COM.keytab.
I20260812 07:03:56.905055 12035 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskpv3LkS/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/postgres -l logfile start
2026-08-12 07:03:59.061 UTC [31709] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 07:03:59.061 UTC [31709] LOG: listening on IPv4 address "127.11.192.212", port 36441
2026-08-12 07:03:59.064 UTC [31709] LOG: listening on Unix socket "/tmp/.s.PGSQL.36441"
2026-08-12 07:03:59.085 UTC [31712] LOG: database system was shut down at 2026-08-12 07:03:57 UTC
2026-08-12 07:03:59.088 UTC [31709] LOG: database system is ready to accept connections
I20260812 07:04:00.213989 12035 mini_postgres.cc:96] Postgres bound to 36441
2026-08-12 07:04:00.238 UTC [31724] FATAL: database "slave" does not exist
127.11.192.212:36441 - accepting connections
I20260812 07:04:00.238864 12035 mini_ranger.cc:161] Starting Ranger...
I20260812 07:04:00.266420 12035 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 07:04:00.647231 12035 mini_ranger.cc:87] Created ranger Postgres database
I20260812 07:04:00.647368 12035 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ranger-admin
2026-08-12 07:04:00,981 [I] DB FLAVOR :POSTGRES
2026-08-12 07:04:00,981 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 07:04:00,981 [I] Checking connection..
2026-08-12 07:04:00,981 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 07:04:01,236 [I] Checking connection passed.
2026-08-12 07:04:01,237 [I] --------- Verifying version history table ---------
2026-08-12 07:04:01,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 07:04:01,491 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 07:04:01,492 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 07:04:01,747 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 07:04:01,747 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 07:04:01,747 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 07:04:01.993 UTC [31835] WARNING: there is no transaction in progress
2026-08-12 07:04:02,002 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 07:04:02,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 07:04:02,261 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 07:04:02,261 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 07:04:02,262 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 07:04:02,511 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 07:04:02,758 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-lgr5.c.gcp-upstream.internal','N') ;"
2026-08-12 07:04:02,987 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 07:04:03,252 [I] Table x_portal_user does not exist in database ranger
2026-08-12 07:04:03,252 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 07:04:03,508 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 07:04:03,508 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 07:04:03,760 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 07:04:04,007 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 07:04:04,007 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 07:04:04.846 UTC [32018] WARNING: there is no transaction in progress
2026-08-12 07:04:04.860 UTC [32018] WARNING: there is no transaction in progress
2026-08-12 07:04:04.874 UTC [32018] WARNING: there is no transaction in progress
2026-08-12 07:04:04.887 UTC [32018] WARNING: there is no transaction in progress
2026-08-12 07:04:04.901 UTC [32018] WARNING: there is no transaction in progress
2026-08-12 07:04:04.917 UTC [32018] WARNING: there is no transaction in progress
2026-08-12 07:04:04.964 UTC [32018] WARNING: there is no transaction in progress
2026-08-12 07:04:04.969 UTC [32018] WARNING: there is no transaction in progress
2026-08-12 07:04:04.982 UTC [32018] WARNING: there is no transaction in progress
2026-08-12 07:04:04.990 UTC [32018] WARNING: there is no transaction in progress
2026-08-12 07:04:04.999 UTC [32018] WARNING: there is no transaction in progress
2026-08-12 07:04:05.012 UTC [32018] WARNING: there is no transaction in progress
2026-08-12 07:04:05.020 UTC [32018] WARNING: there is no transaction in progress
2026-08-12 07:04:05.025 UTC [32018] WARNING: there is no transaction in progress
2026-08-12 07:04:05.031 UTC [32018] WARNING: there is no transaction in progress
2026-08-12 07:04:05.037 UTC [32018] WARNING: there is no transaction in progress
2026-08-12 07:04:05,577 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 07:04:05,578 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 07:04:05,822 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 07:04:05,822 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 07:04:06,058 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 07:04:06,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 07:04:06,304 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 07:04:06,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 07:04:06,554 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 07:04:06,803 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-lgr5.c.gcp-upstream.internal';"
2026-08-12 07:04:07,019 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 07:04:07,019 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 07:04:07,257 [I] DB_PATCHES have already been applied
I20260812 07:04:07.263159 12035 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ranger-admin:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskpv3LkS/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 07:04:07.263263 12035 mini_ranger.cc:193] Using host: 127.11.192.212
I20260812 07:04:07.283505 12035 mini_ranger.cc:227] Ranger admin URL: http://127.11.192.212:39187
Aug 12, 2026 7:04:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 7:04:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 7:04:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 7:04:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 7:04:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 7:04:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 7:04:08 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 07:04:08.563916 12035 mini_ranger.cc:160] Time spent starting Ranger: real 8.325s user 0.000s sys 0.056s
Aug 12, 2026 7:04:27 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 7:04:27 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 7:04:27 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 7:04:27 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 7:04:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 7:04:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 7:04:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 7:04:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 7:04:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 7:04:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 7:04:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 7:04:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 7:04:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 7:04:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 7:04:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 7:04:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 7:04:27 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 7:04:28 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 07:04:29.378878 12035 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.192.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.192.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.192.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/rangerkms_127.11.192.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.192.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/rangerkms_127.11.192.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.192.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.192.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.192.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/HTTP_127.11.192.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.192.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/HTTP_127.11.192.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 07:04:29 dist-test-slave-lgr5 krb5kdc[31651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518269, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 07:04:29.565431 12035 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 07:04:29.592046 12035 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 07:04:30.300585 12035 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 07:04:30.300726 12035 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ranger-kms
I20260812 07:04:30.300870 12035 mini_ranger_kms.cc:227] Using postgres at 127.11.192.212:36441
2026-08-12 07:04:30,842 [I] DB FLAVOR :POSTGRES
2026-08-12 07:04:30,842 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 07:04:30,842 [I] Checking connection
2026-08-12 07:04:30,842 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 07:04:31,067 [I] connection success
2026-08-12 07:04:31,067 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 07:04:31,067 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 07:04:31,068 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 07:04:31,302 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 07:04:31,302 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 07:04:31,302 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 07:04:31,302 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:36441/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 07:04:31,550 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 07:04:32.002856 12035 mini_ranger_kms.cc:310] Created kms service
I20260812 07:04:32.175302 12035 mini_ranger_kms.cc:326] Created kudu user
I20260812 07:04:32.220271 12035 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 07:04:32 dist-test-slave-lgr5 krb5kdc[31651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518269, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
I20260812 07:04:32.613026 12035 mini_ranger_kms.cc:384] Added ranger policy
I20260812 07:04:32.614270 12035 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ranger-kms:/tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 07:04:32.614332 12035 mini_ranger_kms.cc:240] Using host: 127.11.192.212
I20260812 07:04:32.643929 12035 mini_ranger_kms.cc:276] Ranger KMS PID: 32327
I20260812 07:04:32.644104 12035 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.192.212:51287
Aug 12, 2026 7:04:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 7:04:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 7:04:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 7:04:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 7:04:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 7:04:34 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 07:04:34 dist-test-slave-lgr5 krb5kdc[31651](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786518274, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.192.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 07:04:35 dist-test-slave-lgr5 krb5kdc[31651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518274, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.192.212@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
I20260812 07:04:35.329248 12035 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.764s user 0.111s sys 0.468s
I20260812 07:04:35.329613 12035 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 07:04:35.329820 12035 mini_ranger_kms.cc:401] 127.11.192.212:51287/kms/v1/keys
Aug 12 07:04:35 dist-test-slave-lgr5 krb5kdc[31651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518274, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.192.212@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
Aug 12 07:04:37 dist-test-slave-lgr5 krb5kdc[31651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:04:37.953756 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:36727
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:36727
--ranger_config_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.192.212:51287/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:04:38.285557 32430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:04:38.285826 32430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:04:38.285952 32430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:04:38.293025 32430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:04:38.293123 32430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:04:38.293191 32430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:04:38.293262 32430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:04:38.293322 32430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:04:38.303148 32430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.192.212:51287/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:36727
--ranger_config_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:36727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:04:38.304327 32430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:04:38.305596 32430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:04:38.316612 32440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:04:38.316694 32443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:04:38.316707 32441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:04:38.317636 32430 server_base.cc:1034] running on GCE node
I20260812 07:04:38.318342 32430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:04:38.319619 32430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:04:38.320798 32430 hybrid_clock.cc:648] HybridClock initialized: now 1786518278320755 us; error 37 us; skew 500 ppm
I20260812 07:04:38.321134 32430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:04:38 dist-test-slave-lgr5 krb5kdc[31651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518278, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:04:38.324214 32430 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:04:38.325541 32430 webserver.cc:460] Webserver started at http://127.11.192.254:33943/ using document root <none> and password file <none>
I20260812 07:04:38.326120 32430 fs_manager.cc:359] Metadata directory not provided
I20260812 07:04:38.326231 32430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:04:38.326506 32430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:04:38 dist-test-slave-lgr5 krb5kdc[31651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518278, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
I20260812 07:04:38.384944 32430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "00d2e8ea026b413c916e1c2ed9956645"
format_stamp: "Formatted at 2026-08-12 07:04:38 on dist-test-slave-lgr5"
server_key: "d2600b71fd3eac839743c0d588e88e19"
server_key_iv: "6b547ba2b5650fea079def46966822c5"
server_key_version: "kuduclusterkey@0"
I20260812 07:04:38.385798 32430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "00d2e8ea026b413c916e1c2ed9956645"
format_stamp: "Formatted at 2026-08-12 07:04:38 on dist-test-slave-lgr5"
server_key: "d2600b71fd3eac839743c0d588e88e19"
server_key_iv: "6b547ba2b5650fea079def46966822c5"
server_key_version: "kuduclusterkey@0"
I20260812 07:04:38.391186 32430 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:04:38.406629 32457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:04:38.408021 32430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:04:38.408226 32430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "00d2e8ea026b413c916e1c2ed9956645"
format_stamp: "Formatted at 2026-08-12 07:04:38 on dist-test-slave-lgr5"
server_key: "d2600b71fd3eac839743c0d588e88e19"
server_key_iv: "6b547ba2b5650fea079def46966822c5"
server_key_version: "kuduclusterkey@0"
I20260812 07:04:38.408432 32430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:04:38.420688 32430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:04:38.424551 32430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:04:38.424816 32430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:04:38.441866 32430 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:36727
I20260812 07:04:38.441903 32566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:36727 every 8 connection(s)
I20260812 07:04:38.443130 32430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:04:38.447187 32567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:04:38.450651 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 32430
I20260812 07:04:38.450856 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:04:38.451670 12035 external_mini_cluster.cc:1428] Setting key f84a215bd71486a9bd69eaffa2c2a433
I20260812 07:04:38.459400 32567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645: Bootstrap starting.
I20260812 07:04:38.462666 32567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645: Neither blocks nor log segments found. Creating new log.
I20260812 07:04:38.463617 32567 log.cc:824] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645: Log is configured to *not* fsync() on all Append() calls
Aug 12 07:04:38 dist-test-slave-lgr5 krb5kdc[31651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:04:38.466511 32567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645: No bootstrap required, opened a new log
I20260812 07:04:38.474002 32576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:04:38.453616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55626 (local address 127.11.192.254:36727)
0812 07:04:38.454466 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0812 07:04:38.454477 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:04:38.454517 (+ 40us) server_negotiation.cc:407] Connection header received
0812 07:04:38.456218 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:04:38.456241 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:04:38.456591 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:04:38.456951 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:04:38.458236 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:38.459530 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:38.461006 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:38.461675 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:38.461844 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:04:38.465682 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:04:38.465693 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:04:38.465711 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:04:38.465760 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:04:38.469601 (+ 3841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:04:38.470245 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:04:38.470253 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:04:38.470261 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:04:38.470466 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:04:38.470785 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:04:38.470790 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:04:38.470793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:04:38.471464 (+ 671us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:04:38.471771 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0812 07:04:38.472150 (+ 379us) server_negotiation.cc:299] Negotiation successful
0812 07:04:38.472545 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":435,"threads_started":1}
I20260812 07:04:38.476783 32567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d2e8ea026b413c916e1c2ed9956645" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36727 } }
I20260812 07:04:38.477137 32567 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:04:38.477214 32567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d2e8ea026b413c916e1c2ed9956645, State: Initialized, Role: FOLLOWER
I20260812 07:04:38.477712 32567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d2e8ea026b413c916e1c2ed9956645" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36727 } }
I20260812 07:04:38.477936 32567 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:04:38.478042 32567 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:04:38.478201 32567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:04:38.480449 32567 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d2e8ea026b413c916e1c2ed9956645" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36727 } }
I20260812 07:04:38.480906 32567 leader_election.cc:302] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00d2e8ea026b413c916e1c2ed9956645; no voters:
I20260812 07:04:38.481416 32567 leader_election.cc:288] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:04:38.481523 32581 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:04:38.482478 32581 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645 [term 1 LEADER]: Becoming Leader. State: Replica: 00d2e8ea026b413c916e1c2ed9956645, State: Running, Role: LEADER
I20260812 07:04:38.482944 32581 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d2e8ea026b413c916e1c2ed9956645" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36727 } }
I20260812 07:04:38.483316 32567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:04:38.486001 32582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00d2e8ea026b413c916e1c2ed9956645" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d2e8ea026b413c916e1c2ed9956645" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36727 } } }
I20260812 07:04:38.486059 32585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00d2e8ea026b413c916e1c2ed9956645. Latest consensus state: current_term: 1 leader_uuid: "00d2e8ea026b413c916e1c2ed9956645" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d2e8ea026b413c916e1c2ed9956645" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36727 } } }
I20260812 07:04:38.486912 32585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645 [sys.catalog]: This master's current role is: LEADER
I20260812 07:04:38.486912 32582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645 [sys.catalog]: This master's current role is: LEADER
I20260812 07:04:38.488659 32598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:04:38.493120 32598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:04:38.494040 32567 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 07:04:38.518011 32598 catalog_manager.cc:1347] Generated new cluster ID: a9484c67593d46889ba8d51043095f00
I20260812 07:04:38.518147 32598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:04:38.549716 32598 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:04:38.551755 32598 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:04:38.563197 32598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645: Generated new TSK 0
I20260812 07:04:38.564292 32598 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
I20260812 07:04:39.297645 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.192.212:51287/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36727
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:04:39 dist-test-slave-lgr5 krb5kdc[31651](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786518279, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 07:04:39.731456 32659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:04:39.731879 32659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:04:39.732044 32659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:04:39.743577 32659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:04:39.743767 32659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:04:39.743984 32659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:04:39.759641 32659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.192.212:51287/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36727
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:04:39.761467 32659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:04:39.763249 32659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:04:39.779922 32694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:04:39.780126 32683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:04:39.780493 32682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:04:39.789224 32659 server_base.cc:1034] running on GCE node
I20260812 07:04:39.789999 32659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:04:39.791177 32659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:04:39.792412 32659 hybrid_clock.cc:648] HybridClock initialized: now 1786518279792349 us; error 50 us; skew 500 ppm
I20260812 07:04:39.792775 32659 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:04:39 dist-test-slave-lgr5 krb5kdc[31651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518279, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:04:39.796741 32659 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:04:39.813290 32659 webserver.cc:460] Webserver started at http://127.11.192.193:42943/ using document root <none> and password file <none>
I20260812 07:04:39.814424 32659 fs_manager.cc:359] Metadata directory not provided
I20260812 07:04:39.814625 32659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:04:39.815066 32659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:04:39 dist-test-slave-lgr5 krb5kdc[31651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518279, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
I20260812 07:04:39.833967 32659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "90af32f4c5e44d04b321b9ee874041eb"
format_stamp: "Formatted at 2026-08-12 07:04:39 on dist-test-slave-lgr5"
server_key: "fd691eae56cc8a4d6557543142bbbd4c"
server_key_iv: "ef7a6f80161901cbb610777b9bebe809"
server_key_version: "kuduclusterkey@0"
I20260812 07:04:39.835120 32659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "90af32f4c5e44d04b321b9ee874041eb"
format_stamp: "Formatted at 2026-08-12 07:04:39 on dist-test-slave-lgr5"
server_key: "fd691eae56cc8a4d6557543142bbbd4c"
server_key_iv: "ef7a6f80161901cbb610777b9bebe809"
server_key_version: "kuduclusterkey@0"
I20260812 07:04:39.854164 32659 fs_manager.cc:691] Time spent creating directory manager: real 0.018s user 0.009s sys 0.007s
I20260812 07:04:39.884610 32710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:04:39.886704 32659 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.002s sys 0.009s
I20260812 07:04:39.887001 32659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "90af32f4c5e44d04b321b9ee874041eb"
format_stamp: "Formatted at 2026-08-12 07:04:39 on dist-test-slave-lgr5"
server_key: "fd691eae56cc8a4d6557543142bbbd4c"
server_key_iv: "ef7a6f80161901cbb610777b9bebe809"
server_key_version: "kuduclusterkey@0"
I20260812 07:04:39.887367 32659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:04:39.922235 32659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:04:39.926633 32659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:04:39.926921 32659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:04:39.934096 32659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:04:39.936101 32659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:04:39.936201 32659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260812 07:04:39.936328 32659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:04:39.936396 32659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:04:39 dist-test-slave-lgr5 krb5kdc[31651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518279, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
I20260812 07:04:40.264209 464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:38507 every 8 connection(s)
I20260812 07:04:40.264556 32659 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:38507
I20260812 07:04:40.266242 32659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:04:40.268544 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 32659
I20260812 07:04:40.268752 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:04:40.269528 12035 external_mini_cluster.cc:1428] Setting key d74334847ce6a0674f7d7e1b68919766
Aug 12 07:04:40 dist-test-slave-lgr5 krb5kdc[31651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518279, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:04:40.331638 469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:04:40.288494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:36727 (local address 127.11.192.193:42191)
0812 07:04:40.298814 (+ 10320us) negotiation.cc:107] Waiting for socket to connect
0812 07:04:40.298880 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 07:04:40.303713 (+ 4833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:04:40.307752 (+ 4039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:04:40.307764 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:04:40.308469 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:04:40.309306 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:04:40.309334 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:04:40.311051 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:04:40.311057 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:04:40.313285 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:04:40.313300 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:04:40.314308 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:04:40.314315 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:04:40.314567 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:04:40.315316 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:04:40.315368 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:04:40.318339 (+ 2971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:04:40.322064 (+ 3725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:04:40.322072 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:04:40.322086 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:04:40.322609 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:04:40.323036 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:04:40.323041 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:04:40.323044 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:04:40.323290 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:04:40.328896 (+ 5606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:04:40.328904 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:04:40.329335 (+ 431us) client_negotiation.cc:769] Sending connection context
0812 07:04:40.329708 (+ 373us) client_negotiation.cc:240] Negotiation successful
0812 07:04:40.330216 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9986,"thread_start_us":11749,"threads_started":1}
I20260812 07:04:40.332948 468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:04:40.287793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:42191 (local address 127.11.192.254:36727)
0812 07:04:40.307041 (+ 19248us) server_negotiation.cc:206] Beginning negotiation
0812 07:04:40.307047 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:04:40.307069 (+ 22us) server_negotiation.cc:407] Connection header received
0812 07:04:40.307132 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:04:40.307138 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:04:40.307238 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:04:40.309500 (+ 2262us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:04:40.309697 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:40.310835 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:40.313447 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:40.314049 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:40.314962 (+ 913us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:04:40.318600 (+ 3638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:04:40.318607 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:04:40.318612 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:04:40.318648 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:04:40.321913 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:04:40.322764 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:04:40.322768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:04:40.322771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:04:40.322919 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:04:40.323434 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:04:40.323438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:04:40.323441 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:04:40.323764 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:04:40.332180 (+ 8416us) server_negotiation.cc:1035] Waiting for connection context
0812 07:04:40.332318 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 07:04:40.332578 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19108,"thread_start_us":19524,"threads_started":1}
I20260812 07:04:40.334497 465 heartbeater.cc:347] Connected to a master server at 127.11.192.254:36727
I20260812 07:04:40.334831 465 heartbeater.cc:464] Registering TS with master...
I20260812 07:04:40.335695 465 heartbeater.cc:511] Master 127.11.192.254:36727 requested a full tablet report, sending...
I20260812 07:04:40.338269 32495 ts_manager.cc:194] Registered new tserver with Master: 90af32f4c5e44d04b321b9ee874041eb (127.11.192.193:38507)
I20260812 07:04:40.340222 32495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:42191
WARNING: no policy specified for kudu/127.11.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.194@KRBTEST.COM" created.
Aug 12 07:04:40 dist-test-slave-lgr5 krb5kdc[31651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518279, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
W20260812 07:04:40.395008 12035 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.11.192.194: real 0.123s user 0.004s sys 0.020s
I20260812 07:04:40.396107 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.192.212:51287/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36727
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:04:40.814060 475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:04:40.814321 475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:04:40.814484 475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:04:40.821511 475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:04:40.821626 475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:04:40.821791 475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:04:40.831297 475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.192.212:51287/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36727
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:04:40.832670 475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:04:40.833973 475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:04:40.846340 498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:04:40.846419 499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:04:40.846380 501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:04:40.847338 475 server_base.cc:1034] running on GCE node
I20260812 07:04:40.847851 475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:04:40.848750 475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:04:40.849952 475 hybrid_clock.cc:648] HybridClock initialized: now 1786518280849909 us; error 29 us; skew 500 ppm
I20260812 07:04:40.850276 475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:04:40 dist-test-slave-lgr5 krb5kdc[31651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518280, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:04:40.853230 475 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:04:40.854557 475 webserver.cc:460] Webserver started at http://127.11.192.194:32789/ using document root <none> and password file <none>
I20260812 07:04:40.855211 475 fs_manager.cc:359] Metadata directory not provided
I20260812 07:04:40.855329 475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:04:40.855619 475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:04:40 dist-test-slave-lgr5 krb5kdc[31651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518280, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
I20260812 07:04:40.871662 475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "8844894a846f4291a4c3a32f52e24639"
format_stamp: "Formatted at 2026-08-12 07:04:40 on dist-test-slave-lgr5"
server_key: "ea8a70cd3032a1ba1c3f35d636949200"
server_key_iv: "339f442cd332db33c9654d9820e6e546"
server_key_version: "kuduclusterkey@0"
I20260812 07:04:40.872768 475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "8844894a846f4291a4c3a32f52e24639"
format_stamp: "Formatted at 2026-08-12 07:04:40 on dist-test-slave-lgr5"
server_key: "ea8a70cd3032a1ba1c3f35d636949200"
server_key_iv: "339f442cd332db33c9654d9820e6e546"
server_key_version: "kuduclusterkey@0"
I20260812 07:04:40.879597 475 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260812 07:04:40.897564 515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:04:40.899391 475 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260812 07:04:40.899577 475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "8844894a846f4291a4c3a32f52e24639"
format_stamp: "Formatted at 2026-08-12 07:04:40 on dist-test-slave-lgr5"
server_key: "ea8a70cd3032a1ba1c3f35d636949200"
server_key_iv: "339f442cd332db33c9654d9820e6e546"
server_key_version: "kuduclusterkey@0"
I20260812 07:04:40.899814 475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:04:40.926738 475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:04:40.930857 475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:04:40.931142 475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:04:40.932121 475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:04:40.933526 475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:04:40.933605 475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:04:40.933725 475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:04:40.933777 475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:04:40.967036 475 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:40895
I20260812 07:04:40.967129 697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:40895 every 8 connection(s)
I20260812 07:04:40.968354 475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:04:40.977869 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 475
I20260812 07:04:40.978055 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:04:40.978623 12035 external_mini_cluster.cc:1428] Setting key c0a05ae71a188b9036151ffc1cbeb82a
Aug 12 07:04:40 dist-test-slave-lgr5 krb5kdc[31651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518280, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:04:40.993588 703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:04:40.971071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:46657 (local address 127.11.192.254:36727)
0812 07:04:40.971639 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0812 07:04:40.971646 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:04:40.972601 (+ 955us) server_negotiation.cc:407] Connection header received
0812 07:04:40.974417 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:04:40.974423 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:04:40.974531 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:04:40.974659 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:04:40.976013 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:40.977249 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:40.979403 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:40.979971 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:40.980153 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:04:40.986622 (+ 6469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:04:40.986630 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:04:40.986635 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:04:40.986675 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:04:40.989970 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:04:40.990855 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:04:40.990861 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:04:40.990864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:04:40.991007 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:04:40.991502 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:04:40.991507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:04:40.991510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:04:40.991836 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:04:40.991977 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 07:04:40.992967 (+ 990us) server_negotiation.cc:299] Negotiation successful
0812 07:04:40.993199 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":426,"thread_start_us":294,"threads_started":1}
I20260812 07:04:40.995136 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:04:40.971598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:36727 (local address 127.11.192.194:46657)
0812 07:04:40.972432 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0812 07:04:40.972478 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:04:40.974149 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:04:40.974824 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:04:40.974831 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:04:40.975276 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:04:40.975819 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:04:40.975836 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:04:40.977390 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:04:40.977393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:04:40.979260 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:04:40.979272 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:04:40.980206 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:04:40.980214 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:04:40.980480 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:04:40.983711 (+ 3231us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:04:40.983747 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:04:40.986357 (+ 2610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:04:40.990122 (+ 3765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:04:40.990131 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:04:40.990156 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:04:40.990732 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:04:40.991113 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:04:40.991117 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:04:40.991122 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:04:40.991382 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:04:40.991968 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:04:40.991976 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:04:40.992430 (+ 454us) client_negotiation.cc:769] Sending connection context
0812 07:04:40.992859 (+ 429us) client_negotiation.cc:240] Negotiation successful
0812 07:04:40.993393 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":334,"threads_started":1}
I20260812 07:04:40.996881 699 heartbeater.cc:347] Connected to a master server at 127.11.192.254:36727
I20260812 07:04:40.997195 699 heartbeater.cc:464] Registering TS with master...
I20260812 07:04:40.998016 699 heartbeater.cc:511] Master 127.11.192.254:36727 requested a full tablet report, sending...
I20260812 07:04:40.999693 32495 ts_manager.cc:194] Registered new tserver with Master: 8844894a846f4291a4c3a32f52e24639 (127.11.192.194:40895)
I20260812 07:04:41.000864 32495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:46657
WARNING: no policy specified for kudu/127.11.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
I20260812 07:04:41.041450 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.192.212:51287/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36727
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:04:41.343683 465 heartbeater.cc:503] Master 127.11.192.254:36727 was elected leader, sending a full tablet report...
W20260812 07:04:41.375995 713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:04:41.376258 713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:04:41.376350 713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:04:41.383282 713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:04:41.383378 713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:04:41.383499 713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:04:41.392840 713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.192.212:51287/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36727
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:04:41.394012 713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:04:41.395262 713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:04:41.408020 723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:04:41.407922 729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:04:41.408026 724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:04:41.409034 713 server_base.cc:1034] running on GCE node
I20260812 07:04:41.409582 713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:04:41.410459 713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:04:41.411688 713 hybrid_clock.cc:648] HybridClock initialized: now 1786518281411617 us; error 46 us; skew 500 ppm
I20260812 07:04:41.412039 713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:04:41 dist-test-slave-lgr5 krb5kdc[31651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518281, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:04:41.417634 713 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:04:41.419030 713 webserver.cc:460] Webserver started at http://127.11.192.195:36303/ using document root <none> and password file <none>
I20260812 07:04:41.419658 713 fs_manager.cc:359] Metadata directory not provided
I20260812 07:04:41.419773 713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:04:41.420110 713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:04:41 dist-test-slave-lgr5 krb5kdc[31651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518281, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
I20260812 07:04:41.435809 713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "9c8599c44ba840eaba30ba00d51e9b08"
format_stamp: "Formatted at 2026-08-12 07:04:41 on dist-test-slave-lgr5"
server_key: "51b0b9f1f8a86d96a1ff0c5ef151b49e"
server_key_iv: "6bbc2925ecbe0500ac5629f2be67266c"
server_key_version: "kuduclusterkey@0"
I20260812 07:04:41.436865 713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "9c8599c44ba840eaba30ba00d51e9b08"
format_stamp: "Formatted at 2026-08-12 07:04:41 on dist-test-slave-lgr5"
server_key: "51b0b9f1f8a86d96a1ff0c5ef151b49e"
server_key_iv: "6bbc2925ecbe0500ac5629f2be67266c"
server_key_version: "kuduclusterkey@0"
I20260812 07:04:41.443961 713 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260812 07:04:41.458817 740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:04:41.460278 713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:04:41.460428 713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "9c8599c44ba840eaba30ba00d51e9b08"
format_stamp: "Formatted at 2026-08-12 07:04:41 on dist-test-slave-lgr5"
server_key: "51b0b9f1f8a86d96a1ff0c5ef151b49e"
server_key_iv: "6bbc2925ecbe0500ac5629f2be67266c"
server_key_version: "kuduclusterkey@0"
I20260812 07:04:41.460614 713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:04:41.489656 713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:04:41.493088 713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:04:41.493362 713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:04:41.494269 713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:04:41.495607 713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:04:41.495697 713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:04:41.495813 713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:04:41.495874 713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:04:41.525671 713 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:45467
I20260812 07:04:41.525712 940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:45467 every 8 connection(s)
I20260812 07:04:41.526818 713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:04:41.536406 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 713
I20260812 07:04:41.536598 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:04:41.537160 12035 external_mini_cluster.cc:1428] Setting key 7b9a93dbd28247bc8bd52674db7b9eb4
Aug 12 07:04:41 dist-test-slave-lgr5 krb5kdc[31651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518281, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:04:41.544943 944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:04:41.529217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:42443 (local address 127.11.192.254:36727)
0812 07:04:41.529675 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0812 07:04:41.529679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:04:41.530590 (+ 911us) server_negotiation.cc:407] Connection header received
0812 07:04:41.532330 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:04:41.532336 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:04:41.532408 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:04:41.532500 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:04:41.533782 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.534504 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.535919 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.536356 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.536449 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:04:41.540183 (+ 3734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:04:41.540190 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:04:41.540195 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:04:41.540232 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:04:41.542586 (+ 2354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:04:41.543204 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:04:41.543208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:04:41.543210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:04:41.543323 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:04:41.543627 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:04:41.543630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:04:41.543632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:04:41.543872 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:04:41.543986 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 07:04:41.544535 (+ 549us) server_negotiation.cc:299] Negotiation successful
0812 07:04:41.544693 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":222,"threads_started":1}
I20260812 07:04:41.545720 946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:04:41.529776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:36727 (local address 127.11.192.195:42443)
0812 07:04:41.530435 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0812 07:04:41.530477 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 07:04:41.532062 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:04:41.532681 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:04:41.532687 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:04:41.533108 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:04:41.533643 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:04:41.533661 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.534616 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.534619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:04:41.535806 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:04:41.535818 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.536574 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.536583 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:04:41.536863 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:04:41.537551 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:04:41.537581 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:04:41.539940 (+ 2359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:04:41.542702 (+ 2762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:04:41.542708 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:04:41.542718 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:04:41.543122 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:04:41.543397 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:04:41.543400 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:04:41.543403 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:04:41.543555 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:04:41.543958 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:04:41.543962 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:04:41.544251 (+ 289us) client_negotiation.cc:769] Sending connection context
0812 07:04:41.544453 (+ 202us) client_negotiation.cc:240] Negotiation successful
0812 07:04:41.544766 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":266,"threads_started":1}
I20260812 07:04:41.546907 941 heartbeater.cc:347] Connected to a master server at 127.11.192.254:36727
I20260812 07:04:41.547151 941 heartbeater.cc:464] Registering TS with master...
I20260812 07:04:41.547799 941 heartbeater.cc:511] Master 127.11.192.254:36727 requested a full tablet report, sending...
I20260812 07:04:41.549145 32495 ts_manager.cc:194] Registered new tserver with Master: 9c8599c44ba840eaba30ba00d51e9b08 (127.11.192.195:45467)
I20260812 07:04:41.549893 32495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:42443
I20260812 07:04:41.552482 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:04:41.568204 944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:04:41.556791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33544 (local address 127.11.192.254:36727)
0812 07:04:41.556979 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 07:04:41.556985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:04:41.557264 (+ 279us) server_negotiation.cc:407] Connection header received
0812 07:04:41.557381 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:04:41.557385 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:04:41.557451 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:04:41.557523 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:04:41.558811 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.559502 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.560832 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.561232 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.561327 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:04:41.563460 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:04:41.563464 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:04:41.563467 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:04:41.563494 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:04:41.565767 (+ 2273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:04:41.566367 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:04:41.566370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:04:41.566372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:04:41.566512 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:04:41.566878 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:04:41.566883 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:04:41.566886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:04:41.567141 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:04:41.567374 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 07:04:41.567678 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 07:04:41.567884 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 07:04:41.571666 32495 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:04:41.578712 32495 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:04:41.602646 972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:04:41.590573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:45467 (local address 127.0.0.1:44676)
0812 07:04:41.593906 (+ 3333us) negotiation.cc:107] Waiting for socket to connect
0812 07:04:41.593924 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:04:41.594136 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:04:41.595123 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:04:41.595128 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:04:41.595150 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:04:41.595490 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:04:41.595498 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.597417 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.597422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:04:41.599594 (+ 2172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:04:41.599617 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.601716 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.601723 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:04:41.601912 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:04:41.601974 (+ 62us) client_negotiation.cc:769] Sending connection context
0812 07:04:41.602096 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 07:04:41.602268 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3204,"mutex_wait_us":2,"thread_start_us":1934,"threads_started":1}
I20260812 07:04:41.603483 966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:04:41.590499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:38507 (local address 127.0.0.1:44152)
0812 07:04:41.592677 (+ 2178us) negotiation.cc:107] Waiting for socket to connect
0812 07:04:41.592730 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 07:04:41.592903 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:04:41.595797 (+ 2894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:04:41.595801 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:04:41.595829 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:04:41.596299 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:04:41.596310 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.598504 (+ 2194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.598510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:04:41.600361 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:04:41.600373 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.602708 (+ 2335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.602715 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:04:41.602918 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:04:41.602951 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:04:41.603030 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 07:04:41.603149 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1925,"mutex_wait_us":19,"thread_start_us":5269,"threads_started":1}
I20260812 07:04:41.603540 969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:04:41.591106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44676 (local address 127.11.192.195:45467)
0812 07:04:41.592039 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0812 07:04:41.592049 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:04:41.594026 (+ 1977us) server_negotiation.cc:407] Connection header received
0812 07:04:41.594338 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:04:41.594348 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:04:41.594600 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:04:41.594776 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:04:41.595669 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.597274 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.599779 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.601170 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.601353 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:04:41.601564 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 07:04:41.603016 (+ 1452us) server_negotiation.cc:299] Negotiation successful
0812 07:04:41.603175 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":732,"thread_start_us":542,"threads_started":1}
I20260812 07:04:41.603607 965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:04:41.590601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44152 (local address 127.11.192.193:38507)
0812 07:04:41.595198 (+ 4597us) server_negotiation.cc:206] Beginning negotiation
0812 07:04:41.595206 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:04:41.595236 (+ 30us) server_negotiation.cc:407] Connection header received
0812 07:04:41.595338 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:04:41.595345 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:04:41.595567 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:04:41.596457 (+ 890us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:04:41.596662 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.598355 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.600494 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.602447 (+ 1953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.602912 (+ 465us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:04:41.603068 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 07:04:41.603146 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 07:04:41.603310 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4407,"thread_start_us":4225,"threads_started":1}
I20260812 07:04:41.604249 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:04:41.592105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:40895 (local address 127.0.0.1:53688)
0812 07:04:41.596075 (+ 3970us) negotiation.cc:107] Waiting for socket to connect
0812 07:04:41.596115 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 07:04:41.596240 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:04:41.596886 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:04:41.596891 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:04:41.596917 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:04:41.597282 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:04:41.597297 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.598955 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.598961 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:04:41.600870 (+ 1909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:04:41.600886 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.602660 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.602668 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:04:41.603685 (+ 1017us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:04:41.603716 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 07:04:41.603794 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 07:04:41.603901 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3834,"mutex_wait_us":1,"thread_start_us":1538,"threads_started":1}
I20260812 07:04:41.604910 973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:04:41.592344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53688 (local address 127.11.192.194:40895)
0812 07:04:41.594614 (+ 2270us) server_negotiation.cc:206] Beginning negotiation
0812 07:04:41.594623 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:04:41.596376 (+ 1753us) server_negotiation.cc:407] Connection header received
0812 07:04:41.596486 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:04:41.596492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:04:41.596686 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:04:41.596872 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:04:41.597495 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.598823 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.601037 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.602407 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.602578 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:04:41.602734 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 07:04:41.604210 (+ 1476us) server_negotiation.cc:299] Negotiation successful
0812 07:04:41.604465 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1016,"thread_start_us":880,"threads_started":1}
I20260812 07:04:41.607250 335 tablet_service.cc:1467] Processing CreateTablet for tablet e19e417a9db64aed95dae4429a418fbd (DEFAULT_TABLE table=test-table [id=e329ca07b3754c15b8ab246781e53fab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:04:41.607216 818 tablet_service.cc:1467] Processing CreateTablet for tablet e19e417a9db64aed95dae4429a418fbd (DEFAULT_TABLE table=test-table [id=e329ca07b3754c15b8ab246781e53fab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:04:41.608559 335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e19e417a9db64aed95dae4429a418fbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:04:41.608170 588 tablet_service.cc:1467] Processing CreateTablet for tablet e19e417a9db64aed95dae4429a418fbd (DEFAULT_TABLE table=test-table [id=e329ca07b3754c15b8ab246781e53fab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:04:41.608660 818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e19e417a9db64aed95dae4429a418fbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:04:41.609436 588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e19e417a9db64aed95dae4429a418fbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:04:41.620198 974 tablet_bootstrap.cc:492] T e19e417a9db64aed95dae4429a418fbd P 90af32f4c5e44d04b321b9ee874041eb: Bootstrap starting.
I20260812 07:04:41.622304 976 tablet_bootstrap.cc:492] T e19e417a9db64aed95dae4429a418fbd P 9c8599c44ba840eaba30ba00d51e9b08: Bootstrap starting.
I20260812 07:04:41.623592 974 tablet_bootstrap.cc:654] T e19e417a9db64aed95dae4429a418fbd P 90af32f4c5e44d04b321b9ee874041eb: Neither blocks nor log segments found. Creating new log.
I20260812 07:04:41.623615 984 tablet_bootstrap.cc:492] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639: Bootstrap starting.
I20260812 07:04:41.625083 974 log.cc:824] T e19e417a9db64aed95dae4429a418fbd P 90af32f4c5e44d04b321b9ee874041eb: Log is configured to *not* fsync() on all Append() calls
I20260812 07:04:41.625849 976 tablet_bootstrap.cc:654] T e19e417a9db64aed95dae4429a418fbd P 9c8599c44ba840eaba30ba00d51e9b08: Neither blocks nor log segments found. Creating new log.
I20260812 07:04:41.626874 984 tablet_bootstrap.cc:654] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639: Neither blocks nor log segments found. Creating new log.
I20260812 07:04:41.627322 976 log.cc:824] T e19e417a9db64aed95dae4429a418fbd P 9c8599c44ba840eaba30ba00d51e9b08: Log is configured to *not* fsync() on all Append() calls
I20260812 07:04:41.628250 984 log.cc:824] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639: Log is configured to *not* fsync() on all Append() calls
I20260812 07:04:41.629145 974 tablet_bootstrap.cc:492] T e19e417a9db64aed95dae4429a418fbd P 90af32f4c5e44d04b321b9ee874041eb: No bootstrap required, opened a new log
I20260812 07:04:41.629438 974 ts_tablet_manager.cc:1397] T e19e417a9db64aed95dae4429a418fbd P 90af32f4c5e44d04b321b9ee874041eb: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260812 07:04:41.630390 976 tablet_bootstrap.cc:492] T e19e417a9db64aed95dae4429a418fbd P 9c8599c44ba840eaba30ba00d51e9b08: No bootstrap required, opened a new log
I20260812 07:04:41.630659 976 ts_tablet_manager.cc:1397] T e19e417a9db64aed95dae4429a418fbd P 9c8599c44ba840eaba30ba00d51e9b08: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 07:04:41.631458 984 tablet_bootstrap.cc:492] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639: No bootstrap required, opened a new log
I20260812 07:04:41.631789 984 ts_tablet_manager.cc:1397] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 07:04:41.636945 974 raft_consensus.cc:348] T e19e417a9db64aed95dae4429a418fbd P 90af32f4c5e44d04b321b9ee874041eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90af32f4c5e44d04b321b9ee874041eb" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38507 } } peers { permanent_uuid: "9c8599c44ba840eaba30ba00d51e9b08" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 45467 } } peers { permanent_uuid: "8844894a846f4291a4c3a32f52e24639" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 40895 } }
I20260812 07:04:41.637255 974 raft_consensus.cc:374] T e19e417a9db64aed95dae4429a418fbd P 90af32f4c5e44d04b321b9ee874041eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:04:41.637358 974 raft_consensus.cc:729] T e19e417a9db64aed95dae4429a418fbd P 90af32f4c5e44d04b321b9ee874041eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90af32f4c5e44d04b321b9ee874041eb, State: Initialized, Role: FOLLOWER
I20260812 07:04:41.637878 974 consensus_queue.cc:260] T e19e417a9db64aed95dae4429a418fbd P 90af32f4c5e44d04b321b9ee874041eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90af32f4c5e44d04b321b9ee874041eb" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38507 } } peers { permanent_uuid: "9c8599c44ba840eaba30ba00d51e9b08" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 45467 } } peers { permanent_uuid: "8844894a846f4291a4c3a32f52e24639" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 40895 } }
I20260812 07:04:41.639611 974 ts_tablet_manager.cc:1428] T e19e417a9db64aed95dae4429a418fbd P 90af32f4c5e44d04b321b9ee874041eb: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260812 07:04:41.641563 976 raft_consensus.cc:348] T e19e417a9db64aed95dae4429a418fbd P 9c8599c44ba840eaba30ba00d51e9b08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90af32f4c5e44d04b321b9ee874041eb" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38507 } } peers { permanent_uuid: "9c8599c44ba840eaba30ba00d51e9b08" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 45467 } } peers { permanent_uuid: "8844894a846f4291a4c3a32f52e24639" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 40895 } }
I20260812 07:04:41.642163 976 raft_consensus.cc:374] T e19e417a9db64aed95dae4429a418fbd P 9c8599c44ba840eaba30ba00d51e9b08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:04:41.642369 976 raft_consensus.cc:729] T e19e417a9db64aed95dae4429a418fbd P 9c8599c44ba840eaba30ba00d51e9b08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8599c44ba840eaba30ba00d51e9b08, State: Initialized, Role: FOLLOWER
I20260812 07:04:41.642987 976 consensus_queue.cc:260] T e19e417a9db64aed95dae4429a418fbd P 9c8599c44ba840eaba30ba00d51e9b08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90af32f4c5e44d04b321b9ee874041eb" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38507 } } peers { permanent_uuid: "9c8599c44ba840eaba30ba00d51e9b08" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 45467 } } peers { permanent_uuid: "8844894a846f4291a4c3a32f52e24639" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 40895 } }
I20260812 07:04:41.643798 984 raft_consensus.cc:348] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90af32f4c5e44d04b321b9ee874041eb" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38507 } } peers { permanent_uuid: "9c8599c44ba840eaba30ba00d51e9b08" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 45467 } } peers { permanent_uuid: "8844894a846f4291a4c3a32f52e24639" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 40895 } }
I20260812 07:04:41.644125 941 heartbeater.cc:503] Master 127.11.192.254:36727 was elected leader, sending a full tablet report...
I20260812 07:04:41.644263 984 raft_consensus.cc:374] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:04:41.644438 984 raft_consensus.cc:729] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8844894a846f4291a4c3a32f52e24639, State: Initialized, Role: FOLLOWER
I20260812 07:04:41.644917 976 ts_tablet_manager.cc:1428] T e19e417a9db64aed95dae4429a418fbd P 9c8599c44ba840eaba30ba00d51e9b08: Time spent starting tablet: real 0.014s user 0.007s sys 0.009s
I20260812 07:04:41.645027 984 consensus_queue.cc:260] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90af32f4c5e44d04b321b9ee874041eb" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38507 } } peers { permanent_uuid: "9c8599c44ba840eaba30ba00d51e9b08" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 45467 } } peers { permanent_uuid: "8844894a846f4291a4c3a32f52e24639" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 40895 } }
I20260812 07:04:41.646236 699 heartbeater.cc:503] Master 127.11.192.254:36727 was elected leader, sending a full tablet report...
I20260812 07:04:41.646782 984 ts_tablet_manager.cc:1428] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
W20260812 07:04:41.720242 700 tablet.cc:2369] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:04:41.778271 943 tablet.cc:2369] T e19e417a9db64aed95dae4429a418fbd P 9c8599c44ba840eaba30ba00d51e9b08: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:04:41.803371 466 tablet.cc:2369] T e19e417a9db64aed95dae4429a418fbd P 90af32f4c5e44d04b321b9ee874041eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:04:41.813616 998 raft_consensus.cc:482] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:04:41.813972 998 raft_consensus.cc:504] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90af32f4c5e44d04b321b9ee874041eb" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38507 } } peers { permanent_uuid: "9c8599c44ba840eaba30ba00d51e9b08" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 45467 } } peers { permanent_uuid: "8844894a846f4291a4c3a32f52e24639" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 40895 } }
I20260812 07:04:41.815840 998 leader_election.cc:288] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90af32f4c5e44d04b321b9ee874041eb (127.11.192.193:38507), 9c8599c44ba840eaba30ba00d51e9b08 (127.11.192.195:45467)
I20260812 07:04:41.823412 965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:04:41.816388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:45131 (local address 127.11.192.193:38507)
0812 07:04:41.816608 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 07:04:41.816613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:04:41.817193 (+ 580us) server_negotiation.cc:407] Connection header received
0812 07:04:41.817391 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:04:41.817395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:04:41.817455 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:04:41.817536 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:04:41.818096 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.818935 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.821016 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.822531 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.822699 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:04:41.822772 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 07:04:41.823077 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 07:04:41.823175 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:04:41.823827 969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:04:41.816407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:59767 (local address 127.11.192.195:45467)
0812 07:04:41.816603 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 07:04:41.816608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:04:41.817592 (+ 984us) server_negotiation.cc:407] Connection header received
0812 07:04:41.817759 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:04:41.817763 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:04:41.817822 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:04:41.817894 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:04:41.818444 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.819144 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.821201 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.822742 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.822881 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:04:41.822930 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 07:04:41.823278 (+ 348us) server_negotiation.cc:299] Negotiation successful
0812 07:04:41.823399 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 07:04:41.823829 1003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:04:41.816354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:38507 (local address 127.11.192.194:45131)
0812 07:04:41.817080 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0812 07:04:41.817099 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:04:41.817249 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:04:41.817581 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:04:41.817586 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:04:41.817657 (+ 71us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:04:41.817951 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:04:41.817960 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.819073 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.819076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:04:41.820864 (+ 1788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:04:41.820877 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.822755 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.822760 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:04:41.822910 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:04:41.822949 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 07:04:41.823053 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 07:04:41.823185 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"mutex_wait_us":32,"spinlock_wait_cycles":35584,"thread_start_us":436,"threads_started":1}
I20260812 07:04:41.823863 1002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:04:41.816380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:45467 (local address 127.11.192.194:59767)
0812 07:04:41.817489 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0812 07:04:41.817503 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:04:41.817627 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:04:41.817936 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:04:41.817941 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:04:41.817964 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:04:41.818301 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:04:41.818310 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.819286 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.819291 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:04:41.821036 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:04:41.821053 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.822942 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.822948 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:04:41.823124 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:04:41.823155 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 07:04:41.823231 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 07:04:41.823342 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1006,"mutex_wait_us":13,"spinlock_wait_cycles":896,"thread_start_us":820,"threads_started":1}
I20260812 07:04:41.824831 378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e19e417a9db64aed95dae4429a418fbd" candidate_uuid: "8844894a846f4291a4c3a32f52e24639" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90af32f4c5e44d04b321b9ee874041eb" is_pre_election: true
I20260812 07:04:41.825028 858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e19e417a9db64aed95dae4429a418fbd" candidate_uuid: "8844894a846f4291a4c3a32f52e24639" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8599c44ba840eaba30ba00d51e9b08" is_pre_election: true
I20260812 07:04:41.825407 378 raft_consensus.cc:2454] T e19e417a9db64aed95dae4429a418fbd P 90af32f4c5e44d04b321b9ee874041eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8844894a846f4291a4c3a32f52e24639 in term 0.
I20260812 07:04:41.825709 858 raft_consensus.cc:2454] T e19e417a9db64aed95dae4429a418fbd P 9c8599c44ba840eaba30ba00d51e9b08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8844894a846f4291a4c3a32f52e24639 in term 0.
I20260812 07:04:41.826170 516 leader_election.cc:302] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8844894a846f4291a4c3a32f52e24639, 90af32f4c5e44d04b321b9ee874041eb; no voters:
I20260812 07:04:41.826649 998 raft_consensus.cc:2781] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:04:41.826781 998 raft_consensus.cc:482] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:04:41.826964 998 raft_consensus.cc:3037] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:04:41.829622 998 raft_consensus.cc:504] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90af32f4c5e44d04b321b9ee874041eb" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38507 } } peers { permanent_uuid: "9c8599c44ba840eaba30ba00d51e9b08" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 45467 } } peers { permanent_uuid: "8844894a846f4291a4c3a32f52e24639" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 40895 } }
I20260812 07:04:41.830381 998 leader_election.cc:288] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639 [CANDIDATE]: Term 1 election: Requested vote from peers 90af32f4c5e44d04b321b9ee874041eb (127.11.192.193:38507), 9c8599c44ba840eaba30ba00d51e9b08 (127.11.192.195:45467)
I20260812 07:04:41.830802 378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e19e417a9db64aed95dae4429a418fbd" candidate_uuid: "8844894a846f4291a4c3a32f52e24639" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90af32f4c5e44d04b321b9ee874041eb"
I20260812 07:04:41.830813 858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e19e417a9db64aed95dae4429a418fbd" candidate_uuid: "8844894a846f4291a4c3a32f52e24639" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c8599c44ba840eaba30ba00d51e9b08"
I20260812 07:04:41.830964 858 raft_consensus.cc:3037] T e19e417a9db64aed95dae4429a418fbd P 9c8599c44ba840eaba30ba00d51e9b08 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:04:41.831018 378 raft_consensus.cc:3037] T e19e417a9db64aed95dae4429a418fbd P 90af32f4c5e44d04b321b9ee874041eb [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:04:41.833171 858 raft_consensus.cc:2454] T e19e417a9db64aed95dae4429a418fbd P 9c8599c44ba840eaba30ba00d51e9b08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8844894a846f4291a4c3a32f52e24639 in term 1.
I20260812 07:04:41.833539 517 leader_election.cc:302] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8844894a846f4291a4c3a32f52e24639, 9c8599c44ba840eaba30ba00d51e9b08; no voters:
I20260812 07:04:41.833832 998 raft_consensus.cc:2781] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:04:41.834045 378 raft_consensus.cc:2454] T e19e417a9db64aed95dae4429a418fbd P 90af32f4c5e44d04b321b9ee874041eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8844894a846f4291a4c3a32f52e24639 in term 1.
I20260812 07:04:41.834467 998 raft_consensus.cc:686] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639 [term 1 LEADER]: Becoming Leader. State: Replica: 8844894a846f4291a4c3a32f52e24639, State: Running, Role: LEADER
I20260812 07:04:41.835005 998 consensus_queue.cc:237] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90af32f4c5e44d04b321b9ee874041eb" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38507 } } peers { permanent_uuid: "9c8599c44ba840eaba30ba00d51e9b08" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 45467 } } peers { permanent_uuid: "8844894a846f4291a4c3a32f52e24639" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 40895 } }
I20260812 07:04:41.839473 32493 catalog_manager.cc:5495] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8844894a846f4291a4c3a32f52e24639 (127.11.192.194). New cstate: current_term: 1 leader_uuid: "8844894a846f4291a4c3a32f52e24639" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90af32f4c5e44d04b321b9ee874041eb" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c8599c44ba840eaba30ba00d51e9b08" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 45467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8844894a846f4291a4c3a32f52e24639" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 40895 } health_report { overall_health: HEALTHY } } }
I20260812 07:04:41.892338 973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:04:41.886273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53690 (local address 127.11.192.194:40895)
0812 07:04:41.886463 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 07:04:41.886468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:04:41.886516 (+ 48us) server_negotiation.cc:407] Connection header received
0812 07:04:41.886666 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:04:41.886670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:04:41.886747 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:04:41.886879 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:04:41.887254 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.888160 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.890098 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.890614 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.890758 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:04:41.891225 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:04:41.891728 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:04:41.891860 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 07:04:41.891957 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 07:04:41.892061 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:04:41.900578 858 raft_consensus.cc:1261] T e19e417a9db64aed95dae4429a418fbd P 9c8599c44ba840eaba30ba00d51e9b08 [term 1 FOLLOWER]: Refusing update from remote peer 8844894a846f4291a4c3a32f52e24639: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:04:41.900636 378 raft_consensus.cc:1261] T e19e417a9db64aed95dae4429a418fbd P 90af32f4c5e44d04b321b9ee874041eb [term 1 FOLLOWER]: Refusing update from remote peer 8844894a846f4291a4c3a32f52e24639: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:04:41.901562 1004 consensus_queue.cc:1035] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90af32f4c5e44d04b321b9ee874041eb" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:04:41.901862 998 consensus_queue.cc:1035] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c8599c44ba840eaba30ba00d51e9b08" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 45467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:04:41.913857 1015 mvcc.cc:205] Tried to move back new op lower bound from 7317578882654851072 to 7317578882399830016. Current Snapshot: MvccSnapshot[applied={T|T < 7317578882654851072}]
I20260812 07:04:41.928285 32495 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:04:41.928535 32495 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:04:41.932562 32495 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00d2e8ea026b413c916e1c2ed9956645: Sending DeleteTablet for 3 replicas of tablet e19e417a9db64aed95dae4429a418fbd
I20260812 07:04:41.933586 335 tablet_service.cc:1514] Processing DeleteTablet for tablet e19e417a9db64aed95dae4429a418fbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:04:41 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:44152
I20260812 07:04:41.933656 818 tablet_service.cc:1514] Processing DeleteTablet for tablet e19e417a9db64aed95dae4429a418fbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:04:41 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:44676
I20260812 07:04:41.934268 588 tablet_service.cc:1514] Processing DeleteTablet for tablet e19e417a9db64aed95dae4429a418fbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:04:41 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:53688
I20260812 07:04:41.935946 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 32659
I20260812 07:04:41.946106 1034 tablet_replica.cc:330] T e19e417a9db64aed95dae4429a418fbd P 9c8599c44ba840eaba30ba00d51e9b08: stopping tablet replica
I20260812 07:04:41.946508 1034 raft_consensus.cc:2229] T e19e417a9db64aed95dae4429a418fbd P 9c8599c44ba840eaba30ba00d51e9b08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:04:41.947007 1034 raft_consensus.cc:2258] T e19e417a9db64aed95dae4429a418fbd P 9c8599c44ba840eaba30ba00d51e9b08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:04:41.949126 1034 ts_tablet_manager.cc:1905] T e19e417a9db64aed95dae4429a418fbd P 9c8599c44ba840eaba30ba00d51e9b08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:04:41.952600 1035 tablet_replica.cc:330] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639: stopping tablet replica
I20260812 07:04:41.953090 1032 tablet_replica.cc:330] T e19e417a9db64aed95dae4429a418fbd P 90af32f4c5e44d04b321b9ee874041eb: stopping tablet replica
I20260812 07:04:41.953397 1032 raft_consensus.cc:2229] T e19e417a9db64aed95dae4429a418fbd P 90af32f4c5e44d04b321b9ee874041eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:04:41.953774 1032 raft_consensus.cc:2258] T e19e417a9db64aed95dae4429a418fbd P 90af32f4c5e44d04b321b9ee874041eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:04:41.954516 1035 raft_consensus.cc:2229] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:04:41.955186 1035 raft_consensus.cc:2258] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:04:41.956426 1034 ts_tablet_manager.cc:1918] T e19e417a9db64aed95dae4429a418fbd P 9c8599c44ba840eaba30ba00d51e9b08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:04:41.956595 1034 log.cc:1196] T e19e417a9db64aed95dae4429a418fbd P 9c8599c44ba840eaba30ba00d51e9b08: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/e19e417a9db64aed95dae4429a418fbd
I20260812 07:04:41.957160 1034 ts_tablet_manager.cc:1939] T e19e417a9db64aed95dae4429a418fbd P 9c8599c44ba840eaba30ba00d51e9b08: Deleting consensus metadata
I20260812 07:04:41.957873 1032 ts_tablet_manager.cc:1905] T e19e417a9db64aed95dae4429a418fbd P 90af32f4c5e44d04b321b9ee874041eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:04:41.959033 32461 catalog_manager.cc:4841] TS 9c8599c44ba840eaba30ba00d51e9b08 (127.11.192.195:45467): tablet e19e417a9db64aed95dae4429a418fbd (table test-table [id=e329ca07b3754c15b8ab246781e53fab]) successfully deleted
Aug 12 07:04:41 dist-test-slave-lgr5 krb5kdc[31651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.193@KRBTEST.COM
I20260812 07:04:41.962741 1035 ts_tablet_manager.cc:1905] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:04:41.969082 965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:04:41.937187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44160 (local address 127.11.192.193:38507)
0812 07:04:41.937356 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 07:04:41.937362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:04:41.946927 (+ 9565us) server_negotiation.cc:407] Connection header received
0812 07:04:41.947119 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:04:41.947124 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:04:41.947203 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:04:41.947310 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:04:41.948972 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.950235 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.954241 (+ 4006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:41.954869 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:41.955005 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:04:41.961324 (+ 6319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:04:41.961339 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:04:41.961356 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:04:41.961403 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:04:41.965327 (+ 3924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:04:41.966167 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:04:41.966176 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:04:41.966179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:04:41.966385 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:04:41.966848 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:04:41.966853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:04:41.966856 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:04:41.967359 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:04:41.967662 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0812 07:04:41.968375 (+ 713us) server_negotiation.cc:299] Negotiation successful
0812 07:04:41.968592 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:04:41.972644 1032 ts_tablet_manager.cc:1918] T e19e417a9db64aed95dae4429a418fbd P 90af32f4c5e44d04b321b9ee874041eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:04:41.972775 1032 log.cc:1196] T e19e417a9db64aed95dae4429a418fbd P 90af32f4c5e44d04b321b9ee874041eb: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-0/wal/wals/e19e417a9db64aed95dae4429a418fbd
I20260812 07:04:41.973215 1032 ts_tablet_manager.cc:1939] T e19e417a9db64aed95dae4429a418fbd P 90af32f4c5e44d04b321b9ee874041eb: Deleting consensus metadata
I20260812 07:04:41.974869 32460 catalog_manager.cc:4841] TS 90af32f4c5e44d04b321b9ee874041eb (127.11.192.193:38507): tablet e19e417a9db64aed95dae4429a418fbd (table test-table [id=e329ca07b3754c15b8ab246781e53fab]) successfully deleted
I20260812 07:04:41.975566 1035 ts_tablet_manager.cc:1918] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:04:41.975731 1035 log.cc:1196] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/e19e417a9db64aed95dae4429a418fbd
I20260812 07:04:41.976251 1035 ts_tablet_manager.cc:1939] T e19e417a9db64aed95dae4429a418fbd P 8844894a846f4291a4c3a32f52e24639: Deleting consensus metadata
I20260812 07:04:41.977031 457 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:04:41.977993 32462 catalog_manager.cc:4841] TS 8844894a846f4291a4c3a32f52e24639 (127.11.192.194:40895): tablet e19e417a9db64aed95dae4429a418fbd (table test-table [id=e329ca07b3754c15b8ab246781e53fab]) successfully deleted
I20260812 07:04:42.108800 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 32659
I20260812 07:04:42.137110 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 475
Aug 12 07:04:42 dist-test-slave-lgr5 krb5kdc[31651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.194@KRBTEST.COM
I20260812 07:04:42.153524 973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:04:42.137888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53692 (local address 127.11.192.194:40895)
0812 07:04:42.138067 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 07:04:42.138073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:04:42.138145 (+ 72us) server_negotiation.cc:407] Connection header received
0812 07:04:42.138347 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:04:42.138351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:04:42.138426 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:04:42.138518 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:04:42.139941 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:42.140817 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:42.142065 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:42.142517 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:42.142627 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:04:42.146868 (+ 4241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:04:42.146887 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:04:42.146912 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:04:42.146961 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:04:42.150775 (+ 3814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:04:42.151388 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:04:42.151398 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:04:42.151402 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:04:42.151586 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:04:42.151896 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:04:42.151901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:04:42.151905 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:04:42.152386 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:04:42.152745 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0812 07:04:42.152914 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 07:04:42.153145 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 07:04:42.154274 686 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:04:42.266875 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 475
I20260812 07:04:42.289877 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 713
Aug 12 07:04:42 dist-test-slave-lgr5 krb5kdc[31651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.195@KRBTEST.COM
I20260812 07:04:42.304354 969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:04:42.290623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44682 (local address 127.11.192.195:45467)
0812 07:04:42.290797 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 07:04:42.290803 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:04:42.290824 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:04:42.291021 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:04:42.291026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:04:42.291091 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:04:42.291178 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:04:42.292512 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:42.293222 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:42.294539 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:04:42.294925 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:04:42.295052 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:04:42.299129 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:04:42.299143 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:04:42.299159 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:04:42.299198 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:04:42.301646 (+ 2448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:04:42.302295 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:04:42.302302 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:04:42.302304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:04:42.302454 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:04:42.302804 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:04:42.302808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:04:42.302810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:04:42.303178 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:04:42.303445 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0812 07:04:42.303764 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 07:04:42.303962 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:04:42.304904 935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:04:42.391232 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 713
I20260812 07:04:42.411696 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 32430
I20260812 07:04:42.412909 32562 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:04:42.523501 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 32430
I20260812 07:04:42.552418 12035 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 07:04:42.896121 12035 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 07:04:42.896420 12035 mini_ranger.cc:66] Stopping Ranger...
I20260812 07:04:43.240120 12035 mini_ranger.cc:68] Stopped Ranger
2026-08-12 07:04:43.240 UTC [31709] LOG: received smart shutdown request
2026-08-12 07:04:43.243 UTC [31709] LOG: background worker "logical replication launcher" (PID 31722) exited with exit code 1
2026-08-12 07:04:43.243 UTC [31714] LOG: shutting down
2026-08-12 07:04:43.367 UTC [31709] LOG: database system is shut down
2026-08-12T07:04:43Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48142 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:04:43 dist-test-slave-lgr5 krb5kdc[1062](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:04:43 dist-test-slave-lgr5 krb5kdc[1062](info): set up 2 sockets
Aug 12 07:04:43 dist-test-slave-lgr5 krb5kdc[1062](info): commencing operation
krb5kdc: starting...
W20260812 07:04:44.763406 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:04:44 dist-test-slave-lgr5 krb5kdc[1062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:04:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:04:44Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.192.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.192.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.192.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/rangeradmin_127.11.192.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.192.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/rangeradmin_127.11.192.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.192.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.192.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.192.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/rangerlookup_127.11.192.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.192.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/rangerlookup_127.11.192.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.192.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.192.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.192.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/HTTP_127.11.192.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.192.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/HTTP_127.11.192.212@KRBTEST.COM.keytab.
I20260812 07:04:45.072840 12035 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskpv3LkS/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/postgres -l logfile start
2026-08-12 07:04:47.235 UTC [1125] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 07:04:47.235 UTC [1125] LOG: listening on IPv4 address "127.11.192.212", port 60699
2026-08-12 07:04:47.238 UTC [1125] LOG: listening on Unix socket "/tmp/.s.PGSQL.60699"
2026-08-12 07:04:47.258 UTC [1131] LOG: database system was shut down at 2026-08-12 07:04:46 UTC
2026-08-12 07:04:47.261 UTC [1125] LOG: database system is ready to accept connections
I20260812 07:04:48.376588 12035 mini_postgres.cc:96] Postgres bound to 60699
2026-08-12 07:04:48.398 UTC [1150] FATAL: database "slave" does not exist
127.11.192.212:60699 - accepting connections
I20260812 07:04:48.399199 12035 mini_ranger.cc:161] Starting Ranger...
I20260812 07:04:48.426614 12035 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 07:04:48.815583 12035 mini_ranger.cc:87] Created ranger Postgres database
I20260812 07:04:48.815681 12035 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ranger-admin
2026-08-12 07:04:49,105 [I] DB FLAVOR :POSTGRES
2026-08-12 07:04:49,105 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 07:04:49,105 [I] Checking connection..
2026-08-12 07:04:49,105 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 07:04:49,346 [I] Checking connection passed.
2026-08-12 07:04:49,346 [I] --------- Verifying version history table ---------
2026-08-12 07:04:49,346 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 07:04:49,600 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 07:04:49,601 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 07:04:49,855 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 07:04:49,855 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 07:04:49,855 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 07:04:50.087 UTC [1273] WARNING: there is no transaction in progress
2026-08-12 07:04:50,096 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 07:04:50,096 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 07:04:50,349 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 07:04:50,349 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 07:04:50,349 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 07:04:50,602 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 07:04:50,831 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-lgr5.c.gcp-upstream.internal','N') ;"
2026-08-12 07:04:51,069 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 07:04:51,333 [I] Table x_portal_user does not exist in database ranger
2026-08-12 07:04:51,334 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 07:04:51,580 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 07:04:51,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 07:04:51,821 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 07:04:52,075 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 07:04:52,075 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 07:04:52.932 UTC [1461] WARNING: there is no transaction in progress
2026-08-12 07:04:52.946 UTC [1461] WARNING: there is no transaction in progress
2026-08-12 07:04:52.960 UTC [1461] WARNING: there is no transaction in progress
2026-08-12 07:04:52.972 UTC [1461] WARNING: there is no transaction in progress
2026-08-12 07:04:52.986 UTC [1461] WARNING: there is no transaction in progress
2026-08-12 07:04:53.001 UTC [1461] WARNING: there is no transaction in progress
2026-08-12 07:04:53.061 UTC [1461] WARNING: there is no transaction in progress
2026-08-12 07:04:53.068 UTC [1461] WARNING: there is no transaction in progress
2026-08-12 07:04:53.080 UTC [1461] WARNING: there is no transaction in progress
2026-08-12 07:04:53.089 UTC [1461] WARNING: there is no transaction in progress
2026-08-12 07:04:53.097 UTC [1461] WARNING: there is no transaction in progress
2026-08-12 07:04:53.108 UTC [1461] WARNING: there is no transaction in progress
2026-08-12 07:04:53.117 UTC [1461] WARNING: there is no transaction in progress
2026-08-12 07:04:53.123 UTC [1461] WARNING: there is no transaction in progress
2026-08-12 07:04:53.129 UTC [1461] WARNING: there is no transaction in progress
2026-08-12 07:04:53.134 UTC [1461] WARNING: there is no transaction in progress
2026-08-12 07:04:53,664 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 07:04:53,664 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 07:04:53,891 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 07:04:53,892 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 07:04:54,121 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 07:04:54,121 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 07:04:54,367 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 07:04:54,368 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 07:04:54,625 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 07:04:54,874 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-lgr5.c.gcp-upstream.internal';"
2026-08-12 07:04:55,120 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 07:04:55,120 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 07:04:55,376 [I] DB_PATCHES have already been applied
I20260812 07:04:55.382737 12035 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ranger-admin:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskpv3LkS/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 07:04:55.382820 12035 mini_ranger.cc:193] Using host: 127.11.192.212
I20260812 07:04:55.401319 12035 mini_ranger.cc:227] Ranger admin URL: http://127.11.192.212:35573
Aug 12, 2026 7:04:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 7:04:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 7:04:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 7:04:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 7:04:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 7:04:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 7:04:56 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 07:04:56.717670 12035 mini_ranger.cc:160] Time spent starting Ranger: real 8.318s user 0.000s sys 0.051s
Aug 12, 2026 7:05:14 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 7:05:14 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 7:05:14 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 7:05:14 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 7:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 7:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 7:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 7:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 7:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 7:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 7:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 7:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 7:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 7:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 7:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 7:05:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 7:05:14 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 7:05:15 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 07:05:16.329282 12035 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.192.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.192.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.192.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/rangerkms_127.11.192.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.192.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/rangerkms_127.11.192.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.192.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.192.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.192.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/HTTP_127.11.192.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.192.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/HTTP_127.11.192.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 07:05:16 dist-test-slave-lgr5 krb5kdc[1062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518316, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 07:05:16.497884 12035 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 07:05:16.517076 12035 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 07:05:17.217674 12035 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 07:05:17.217765 12035 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ranger-kms
I20260812 07:05:17.217793 12035 mini_ranger_kms.cc:227] Using postgres at 127.11.192.212:60699
2026-08-12 07:05:17,548 [I] DB FLAVOR :POSTGRES
2026-08-12 07:05:17,549 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 07:05:17,549 [I] Checking connection
2026-08-12 07:05:17,549 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 07:05:17,784 [I] connection success
2026-08-12 07:05:17,784 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 07:05:17,784 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 07:05:17,784 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 07:05:18,013 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 07:05:18,013 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 07:05:18,013 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 07:05:18,013 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.192.212:60699/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 07:05:18,255 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 07:05:18.687608 12035 mini_ranger_kms.cc:310] Created kms service
I20260812 07:05:18.854089 12035 mini_ranger_kms.cc:326] Created kudu user
I20260812 07:05:18.894202 12035 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 07:05:18 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518316, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
I20260812 07:05:19.273666 12035 mini_ranger_kms.cc:384] Added ranger policy
I20260812 07:05:19.273963 12035 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ranger-kms:/tmp/dist-test-taskpv3LkS/build/asan/bin/postgresql.jar:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskpv3LkS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 07:05:19.274017 12035 mini_ranger_kms.cc:240] Using host: 127.11.192.212
I20260812 07:05:19.298643 12035 mini_ranger_kms.cc:276] Ranger KMS PID: 1828
I20260812 07:05:19.298758 12035 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.192.212:42425
Aug 12, 2026 7:05:20 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 7:05:20 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 7:05:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 7:05:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 7:05:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 7:05:20 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 07:05:21 dist-test-slave-lgr5 krb5kdc[1062](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786518321, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.192.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 07:05:21 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518321, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.192.212@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
I20260812 07:05:21.950721 12035 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.453s user 0.077s sys 0.280s
I20260812 07:05:21.950976 12035 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 07:05:21.951156 12035 mini_ranger_kms.cc:401] 127.11.192.212:42425/kms/v1/keys
Aug 12 07:05:22 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518321, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.192.212@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
Aug 12 07:05:24 dist-test-slave-lgr5 krb5kdc[1062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:05:24.588797 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:35893
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:42129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:35893
--ranger_config_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.192.212:42425/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:05:24.905699 1947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:24.905956 1947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:24.906049 1947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:24.912911 1947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:05:24.913004 1947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:24.913055 1947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:24.913110 1947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:05:24.913156 1947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:05:24.922746 1947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:42129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.192.212:42425/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:35893
--ranger_config_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:35893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:24.924247 1947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:24.925577 1947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:05:24.936407 1960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:24.936450 1958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:24.936477 1957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:24.937342 1947 server_base.cc:1034] running on GCE node
I20260812 07:05:24.938030 1947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:24.939245 1947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:24.940431 1947 hybrid_clock.cc:648] HybridClock initialized: now 1786518324940385 us; error 39 us; skew 500 ppm
I20260812 07:05:24.940755 1947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:24 dist-test-slave-lgr5 krb5kdc[1062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518324, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:24.950807 1947 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:05:24.952198 1947 webserver.cc:460] Webserver started at http://127.11.192.254:34141/ using document root <none> and password file <none>
I20260812 07:05:24.952766 1947 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:24.952862 1947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:05:24.953112 1947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:05:24 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518324, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
I20260812 07:05:25.007746 1947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "39531b6377634af38c86601f78b25fe9"
format_stamp: "Formatted at 2026-08-12 07:05:25 on dist-test-slave-lgr5"
server_key: "3ae9b3b77b4aa363bdc34648b41c1ab9"
server_key_iv: "94fa494caf77f00fc305d644780d210e"
server_key_version: "kuduclusterkey@0"
I20260812 07:05:25.008692 1947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "39531b6377634af38c86601f78b25fe9"
format_stamp: "Formatted at 2026-08-12 07:05:25 on dist-test-slave-lgr5"
server_key: "3ae9b3b77b4aa363bdc34648b41c1ab9"
server_key_iv: "94fa494caf77f00fc305d644780d210e"
server_key_version: "kuduclusterkey@0"
I20260812 07:05:25.013919 1947 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:05:25.032961 1970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:25.034541 1947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:05:25.034736 1947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "39531b6377634af38c86601f78b25fe9"
format_stamp: "Formatted at 2026-08-12 07:05:25 on dist-test-slave-lgr5"
server_key: "3ae9b3b77b4aa363bdc34648b41c1ab9"
server_key_iv: "94fa494caf77f00fc305d644780d210e"
server_key_version: "kuduclusterkey@0"
I20260812 07:05:25.035009 1947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:25.073185 1947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:25.076790 1947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:25.077054 1947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:25.093595 1947 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:35893
I20260812 07:05:25.093624 2061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:35893 every 8 connection(s)
I20260812 07:05:25.094802 1947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:05:25.098367 2063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:05:25.102773 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 1947
I20260812 07:05:25.102958 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:05:25.103582 12035 external_mini_cluster.cc:1428] Setting key 10c3999d5160894997e96c629e363093
I20260812 07:05:25.108884 2063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9: Bootstrap starting.
I20260812 07:05:25.112356 2063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9: Neither blocks nor log segments found. Creating new log.
I20260812 07:05:25.113523 2063 log.cc:824] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9: Log is configured to *not* fsync() on all Append() calls
I20260812 07:05:25.116714 2063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9: No bootstrap required, opened a new log
Aug 12 07:05:25 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:05:25.124432 2063 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39531b6377634af38c86601f78b25fe9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35893 } }
I20260812 07:05:25.124801 2063 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:05:25.124889 2063 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39531b6377634af38c86601f78b25fe9, State: Initialized, Role: FOLLOWER
I20260812 07:05:25.125470 2063 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39531b6377634af38c86601f78b25fe9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35893 } }
I20260812 07:05:25.125679 2063 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:05:25.125767 2068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:25.105459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36432 (local address 127.11.192.254:35893)
0812 07:05:25.106273 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:25.106287 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:25.106331 (+ 44us) server_negotiation.cc:407] Connection header received
0812 07:05:25.108291 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:25.108313 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:25.108659 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:25.108946 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:25.110513 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:25.111797 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:25.113650 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:25.114589 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:25.114766 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:25.118943 (+ 4177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:25.118950 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:25.118963 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:25.119003 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:25.121951 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:25.122575 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:25.122582 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:25.122587 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:25.122757 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:25.123108 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:25.123112 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:25.123114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:25.123648 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:25.123926 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:25.124365 (+ 439us) server_negotiation.cc:299] Negotiation successful
0812 07:05:25.124723 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":371,"threads_started":1}
I20260812 07:05:25.125818 2063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:05:25.126009 2063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:05:25.128155 2063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39531b6377634af38c86601f78b25fe9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35893 } }
I20260812 07:05:25.128629 2063 leader_election.cc:302] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39531b6377634af38c86601f78b25fe9; no voters:
I20260812 07:05:25.129087 2063 leader_election.cc:288] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:05:25.129194 2070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:05:25.130514 2070 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [term 1 LEADER]: Becoming Leader. State: Replica: 39531b6377634af38c86601f78b25fe9, State: Running, Role: LEADER
I20260812 07:05:25.131042 2063 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:05:25.131616 2070 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39531b6377634af38c86601f78b25fe9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35893 } }
I20260812 07:05:25.134848 2071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39531b6377634af38c86601f78b25fe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39531b6377634af38c86601f78b25fe9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35893 } } }
I20260812 07:05:25.134821 2072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39531b6377634af38c86601f78b25fe9. Latest consensus state: current_term: 1 leader_uuid: "39531b6377634af38c86601f78b25fe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39531b6377634af38c86601f78b25fe9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35893 } } }
I20260812 07:05:25.135294 2071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [sys.catalog]: This master's current role is: LEADER
I20260812 07:05:25.135296 2072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [sys.catalog]: This master's current role is: LEADER
I20260812 07:05:25.136857 2088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:05:25.141734 2088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:05:25.142283 2063 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 07:05:25.162453 2088 catalog_manager.cc:1347] Generated new cluster ID: 57e39b20137c4dca8667115108524cf0
I20260812 07:05:25.162604 2088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:05:25.182356 2088 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:05:25.184254 2088 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:05:25.206568 2088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9: Generated new TSK 0
I20260812 07:05:25.207587 2088 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
I20260812 07:05:25.904521 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.192.212:42425/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:35893
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:42129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:05:26 dist-test-slave-lgr5 krb5kdc[1062](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786518326, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 07:05:26.303923 2151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:26.304239 2151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:26.304381 2151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:26.311769 2151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:26.311877 2151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:26.312032 2151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:05:26.322789 2151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:42129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.192.212:42425/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:35893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:26.324187 2151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:26.325615 2151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:05:26.343637 2171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:26.345363 2174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:26.353681 2151 server_base.cc:1034] running on GCE node
W20260812 07:05:26.354275 2180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:26.354934 2151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:26.356001 2151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:26.357287 2151 hybrid_clock.cc:648] HybridClock initialized: now 1786518326357223 us; error 62 us; skew 500 ppm
I20260812 07:05:26.357613 2151 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:26 dist-test-slave-lgr5 krb5kdc[1062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518326, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:26.360617 2151 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:05:26.362414 2151 webserver.cc:460] Webserver started at http://127.11.192.193:39129/ using document root <none> and password file <none>
I20260812 07:05:26.363099 2151 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:26.363219 2151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:05:26.363526 2151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:05:26 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518326, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
I20260812 07:05:26.379619 2151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "5935e032929a495da3d36d526b0ab0ed"
format_stamp: "Formatted at 2026-08-12 07:05:26 on dist-test-slave-lgr5"
server_key: "f7646cb30619358f919a7ad717382235"
server_key_iv: "5bd9be268866e5e3e753f1498625f00a"
server_key_version: "kuduclusterkey@0"
I20260812 07:05:26.380931 2151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "5935e032929a495da3d36d526b0ab0ed"
format_stamp: "Formatted at 2026-08-12 07:05:26 on dist-test-slave-lgr5"
server_key: "f7646cb30619358f919a7ad717382235"
server_key_iv: "5bd9be268866e5e3e753f1498625f00a"
server_key_version: "kuduclusterkey@0"
I20260812 07:05:26.396533 2151 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.010s sys 0.004s
I20260812 07:05:26.431531 2194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:26.433483 2151 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.005s sys 0.000s
I20260812 07:05:26.433701 2151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "5935e032929a495da3d36d526b0ab0ed"
format_stamp: "Formatted at 2026-08-12 07:05:26 on dist-test-slave-lgr5"
server_key: "f7646cb30619358f919a7ad717382235"
server_key_iv: "5bd9be268866e5e3e753f1498625f00a"
server_key_version: "kuduclusterkey@0"
I20260812 07:05:26.434032 2151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:26.463277 2151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:26.468314 2151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:26.468669 2151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:26.470625 2151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:05:26.476318 2151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:05:26.476483 2151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:26.476677 2151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:05:26.476805 2151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:26.518375 2151 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:40479
I20260812 07:05:26.518397 2415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:40479 every 8 connection(s)
I20260812 07:05:26.519752 2151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:05:26.523377 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 2151
I20260812 07:05:26.523577 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:05:26.524518 12035 external_mini_cluster.cc:1428] Setting key dd4e46992c331fa5bbb050fd3d12081f
Aug 12 07:05:26 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518326, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:05:26.548588 2418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:26.522875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:35895 (local address 127.11.192.254:35893)
0812 07:05:26.525630 (+ 2755us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:26.525638 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:26.528627 (+ 2989us) server_negotiation.cc:407] Connection header received
0812 07:05:26.531018 (+ 2391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:26.531024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:26.531109 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:26.531198 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:26.534027 (+ 2829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:26.535132 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:26.537199 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:26.537736 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:26.537825 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:26.541986 (+ 4161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:26.541993 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:26.541998 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:26.542035 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:26.545310 (+ 3275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:26.545978 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:26.545983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:26.545986 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:26.546124 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:26.546642 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:26.546647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:26.546650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:26.546973 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:26.547108 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:26.547982 (+ 874us) server_negotiation.cc:299] Negotiation successful
0812 07:05:26.548229 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2602,"thread_start_us":7926,"threads_started":1}
I20260812 07:05:26.549733 2419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:26.523309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:35893 (local address 127.11.192.193:35895)
0812 07:05:26.526178 (+ 2869us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:26.526225 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:26.530604 (+ 4379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:26.531635 (+ 1031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:26.531647 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:26.532209 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:05:26.533811 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:26.533848 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:26.535314 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:26.535319 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:26.537049 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:26.537065 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:26.538149 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:26.538156 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:26.538406 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:26.539137 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:05:26.539166 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:05:26.541753 (+ 2587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:05:26.545435 (+ 3682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:26.545444 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:26.545459 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:26.545855 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:26.546249 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:26.546254 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:26.546258 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:26.546501 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:26.547116 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:05:26.547124 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:05:26.547518 (+ 394us) client_negotiation.cc:769] Sending connection context
0812 07:05:26.547834 (+ 316us) client_negotiation.cc:240] Negotiation successful
0812 07:05:26.548384 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2583,"thread_start_us":2354,"threads_started":1}
I20260812 07:05:26.551658 2416 heartbeater.cc:347] Connected to a master server at 127.11.192.254:35893
I20260812 07:05:26.552018 2416 heartbeater.cc:464] Registering TS with master...
I20260812 07:05:26.552920 2416 heartbeater.cc:511] Master 127.11.192.254:35893 requested a full tablet report, sending...
I20260812 07:05:26.555539 2004 ts_manager.cc:194] Registered new tserver with Master: 5935e032929a495da3d36d526b0ab0ed (127.11.192.193:40479)
I20260812 07:05:26.557621 2004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:35895
WARNING: no policy specified for kudu/127.11.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
I20260812 07:05:26.591183 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.192.212:42425/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:35893
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:42129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:05:26 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518326, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
W20260812 07:05:27.009225 2425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:27.009518 2425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:27.009682 2425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:27.018822 2425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:27.019002 2425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:27.019199 2425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
Aug 12 07:05:27 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518326, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
I20260812 07:05:27.034283 2425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:42129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.192.212:42425/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:35893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:27.036007 2425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:27.037801 2425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:05:27.058815 2440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:27.058949 2439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:27.059404 2443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:27.059888 2425 server_base.cc:1034] running on GCE node
I20260812 07:05:27.060613 2425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:27.061751 2425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:27.065109 2425 hybrid_clock.cc:648] HybridClock initialized: now 1786518327064887 us; error 194 us; skew 500 ppm
I20260812 07:05:27.065518 2425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:27 dist-test-slave-lgr5 krb5kdc[1062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518327, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:27.069585 2425 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:05:27.081333 2425 webserver.cc:460] Webserver started at http://127.11.192.194:37383/ using document root <none> and password file <none>
I20260812 07:05:27.082054 2425 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:27.082180 2425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:05:27.082506 2425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:05:27 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518327, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
I20260812 07:05:27.104310 2425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "45e457a2ac3947068168483529c63743"
format_stamp: "Formatted at 2026-08-12 07:05:27 on dist-test-slave-lgr5"
server_key: "bba5e56a2ab0a383ec3228ea7ddd3291"
server_key_iv: "feb008780210150335d4d622637d3b60"
server_key_version: "kuduclusterkey@0"
I20260812 07:05:27.105520 2425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "45e457a2ac3947068168483529c63743"
format_stamp: "Formatted at 2026-08-12 07:05:27 on dist-test-slave-lgr5"
server_key: "bba5e56a2ab0a383ec3228ea7ddd3291"
server_key_iv: "feb008780210150335d4d622637d3b60"
server_key_version: "kuduclusterkey@0"
I20260812 07:05:27.128304 2425 fs_manager.cc:691] Time spent creating directory manager: real 0.022s user 0.002s sys 0.009s
I20260812 07:05:27.180831 2467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:27.186467 2425 fs_manager.cc:725] Time spent opening block manager: real 0.029s user 0.004s sys 0.002s
I20260812 07:05:27.186789 2425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "45e457a2ac3947068168483529c63743"
format_stamp: "Formatted at 2026-08-12 07:05:27 on dist-test-slave-lgr5"
server_key: "bba5e56a2ab0a383ec3228ea7ddd3291"
server_key_iv: "feb008780210150335d4d622637d3b60"
server_key_version: "kuduclusterkey@0"
I20260812 07:05:27.187198 2425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:27.275635 2425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:27.293738 2425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:27.294203 2425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:27.295630 2425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:05:27.304962 2425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:05:27.305132 2425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260812 07:05:27.305310 2425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:05:27.305425 2425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:27.444579 2425 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:36327
I20260812 07:05:27.444609 2727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:36327 every 8 connection(s)
I20260812 07:05:27.446360 2425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:05:27.448612 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 2425
I20260812 07:05:27.448791 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:05:27.449385 12035 external_mini_cluster.cc:1428] Setting key 918fcf40009a89a9c61802c057f718bb
Aug 12 07:05:27 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518327, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:05:27.491014 2732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:27.449307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:57347 (local address 127.11.192.254:35893)
0812 07:05:27.460516 (+ 11209us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:27.460524 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:27.468947 (+ 8423us) server_negotiation.cc:407] Connection header received
0812 07:05:27.471621 (+ 2674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:27.471627 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:27.471719 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:27.471806 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:27.474154 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:27.475409 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:27.477909 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:27.478554 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:27.478650 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:27.483177 (+ 4527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:27.483184 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:27.483189 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:27.483230 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:27.487083 (+ 3853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:27.488050 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:27.488056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:27.488059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:27.488240 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:27.488795 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:27.488801 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:27.488804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:27.489180 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:27.489315 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:27.490337 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0812 07:05:27.490599 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11067,"thread_start_us":10927,"threads_started":1}
WARNING: no policy specified for kudu/127.11.192.195@KRBTEST.COM; defaulting to no policy
I20260812 07:05:27.493628 2733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:27.449923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:35893 (local address 127.11.192.194:57347)
0812 07:05:27.468739 (+ 18816us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:27.468797 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:27.471267 (+ 2470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:27.472426 (+ 1159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:27.472438 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:27.473076 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:05:27.473914 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:27.473950 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:27.475629 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:27.475635 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:27.477741 (+ 2106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:27.477758 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:27.478999 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:27.479007 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:27.479271 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:27.480018 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:05:27.480048 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:05:27.482916 (+ 2868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:05:27.487276 (+ 4360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:27.487285 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:27.487302 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:27.487907 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:27.488404 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:27.488408 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:27.488411 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:27.488657 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:27.489429 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:05:27.489438 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:05:27.489877 (+ 439us) client_negotiation.cc:769] Sending connection context
0812 07:05:27.491604 (+ 1727us) client_negotiation.cc:240] Negotiation successful
0812 07:05:27.492165 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18499,"thread_start_us":18252,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.195@KRBTEST.COM" created.
I20260812 07:05:27.495469 2730 heartbeater.cc:347] Connected to a master server at 127.11.192.254:35893
I20260812 07:05:27.495801 2730 heartbeater.cc:464] Registering TS with master...
I20260812 07:05:27.496707 2730 heartbeater.cc:511] Master 127.11.192.254:35893 requested a full tablet report, sending...
I20260812 07:05:27.498448 2004 ts_manager.cc:194] Registered new tserver with Master: 45e457a2ac3947068168483529c63743 (127.11.192.194:36327)
I20260812 07:05:27.499598 2004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:57347
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
I20260812 07:05:27.531806 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.192.212:42425/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:35893
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:42129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:05:27.560837 2416 heartbeater.cc:503] Master 127.11.192.254:35893 was elected leader, sending a full tablet report...
W20260812 07:05:27.919595 2740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:27.919875 2740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:27.920020 2740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:27.927209 2740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:27.927346 2740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:27.927551 2740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:05:27.937506 2740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:42129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.192.212:42425/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:35893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:27.938815 2740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:27.940162 2740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:05:27.952297 2751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:27.952436 2750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:27.952486 2753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:27.953223 2740 server_base.cc:1034] running on GCE node
I20260812 07:05:27.953758 2740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:27.954602 2740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:27.955789 2740 hybrid_clock.cc:648] HybridClock initialized: now 1786518327955746 us; error 35 us; skew 500 ppm
I20260812 07:05:27.956103 2740 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:27 dist-test-slave-lgr5 krb5kdc[1062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518327, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:27.958945 2740 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:05:27.960279 2740 webserver.cc:460] Webserver started at http://127.11.192.195:33195/ using document root <none> and password file <none>
I20260812 07:05:27.960927 2740 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:27.961058 2740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:05:27.961344 2740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:05:27 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518327, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
I20260812 07:05:27.977722 2740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "65a743ddefbc40fbb298aec04d085a7e"
format_stamp: "Formatted at 2026-08-12 07:05:27 on dist-test-slave-lgr5"
server_key: "717aaf65fb3123529663080df966badc"
server_key_iv: "6e3ea5073a1204ef8b4809bb30960fa2"
server_key_version: "kuduclusterkey@0"
I20260812 07:05:27.978826 2740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "65a743ddefbc40fbb298aec04d085a7e"
format_stamp: "Formatted at 2026-08-12 07:05:27 on dist-test-slave-lgr5"
server_key: "717aaf65fb3123529663080df966badc"
server_key_iv: "6e3ea5073a1204ef8b4809bb30960fa2"
server_key_version: "kuduclusterkey@0"
I20260812 07:05:27.984813 2740 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 07:05:28.000317 2766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:28.001924 2740 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:05:28.002116 2740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "65a743ddefbc40fbb298aec04d085a7e"
format_stamp: "Formatted at 2026-08-12 07:05:27 on dist-test-slave-lgr5"
server_key: "717aaf65fb3123529663080df966badc"
server_key_iv: "6e3ea5073a1204ef8b4809bb30960fa2"
server_key_version: "kuduclusterkey@0"
I20260812 07:05:28.002348 2740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:28.021940 2740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:28.025352 2740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:28.025621 2740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:28.026525 2740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:05:28.027807 2740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:05:28.027890 2740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:28.028018 2740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:05:28.028074 2740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:28.057981 2740 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:40087
I20260812 07:05:28.058024 2979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:40087 every 8 connection(s)
I20260812 07:05:28.059150 2740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:05:28.062248 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 2740
I20260812 07:05:28.062408 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:05:28.062847 12035 external_mini_cluster.cc:1428] Setting key 5b50854fd11b0978bc492227d34c90f6
Aug 12 07:05:28 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518327, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:05:28.077909 2982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:28.061492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:33959 (local address 127.11.192.254:35893)
0812 07:05:28.061938 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:28.061945 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:28.063011 (+ 1066us) server_negotiation.cc:407] Connection header received
0812 07:05:28.064968 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:28.064974 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:28.065085 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:28.065186 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:28.066542 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:28.067246 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:28.068740 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:28.069148 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:28.069237 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:28.072147 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:28.072152 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:28.072155 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:28.072186 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:28.074672 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:28.075465 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:28.075469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:28.075472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:28.075605 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:28.076034 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:28.076038 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:28.076041 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:28.076436 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:28.076570 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:28.077403 (+ 833us) server_negotiation.cc:299] Negotiation successful
0812 07:05:28.077617 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":199,"threads_started":1}
I20260812 07:05:28.078856 2985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:28.061960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:35893 (local address 127.11.192.195:33959)
0812 07:05:28.062766 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:28.062812 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:28.064668 (+ 1856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:28.065379 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:28.065386 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:28.065847 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:05:28.066378 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:28.066396 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:28.067376 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:28.067379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:28.068633 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:28.068646 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:28.069313 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:28.069318 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:28.069507 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:28.070036 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:05:28.070057 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:05:28.071947 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:05:28.074831 (+ 2884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:28.074839 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:28.074849 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:28.075326 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:28.075703 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:28.075706 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:28.075708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:28.075918 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:28.076564 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:05:28.076570 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:05:28.076944 (+ 374us) client_negotiation.cc:769] Sending connection context
0812 07:05:28.077250 (+ 306us) client_negotiation.cc:240] Negotiation successful
0812 07:05:28.077709 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":324,"threads_started":1}
I20260812 07:05:28.080040 2980 heartbeater.cc:347] Connected to a master server at 127.11.192.254:35893
I20260812 07:05:28.080369 2980 heartbeater.cc:464] Registering TS with master...
I20260812 07:05:28.081032 2980 heartbeater.cc:511] Master 127.11.192.254:35893 requested a full tablet report, sending...
I20260812 07:05:28.082398 2004 ts_manager.cc:194] Registered new tserver with Master: 65a743ddefbc40fbb298aec04d085a7e (127.11.192.195:40087)
I20260812 07:05:28.083201 2004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:33959
I20260812 07:05:28.090333 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:05:28.092015 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 2151
Aug 12 07:05:28 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.193@KRBTEST.COM
I20260812 07:05:28.109665 2988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:28.093003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46942 (local address 127.11.192.193:40479)
0812 07:05:28.093679 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:28.093686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:28.093712 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:05:28.093778 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:28.093784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:28.094012 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:28.094142 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:28.096027 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:28.097084 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:28.099146 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:28.099671 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:28.099768 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:28.104575 (+ 4807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:28.104587 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:28.104598 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:28.104634 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:28.107355 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:28.107999 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:28.108004 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:28.108006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:28.108180 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:28.108464 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:28.108467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:28.108469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:28.108785 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:28.109010 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:28.109206 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 07:05:28.109389 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":349,"threads_started":1}
I20260812 07:05:28.110777 2402 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:05:28.192291 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 2151
I20260812 07:05:28.221786 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 2425
Aug 12 07:05:28 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.194@KRBTEST.COM
I20260812 07:05:28.236982 2990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:28.222656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51248 (local address 127.11.192.194:36327)
0812 07:05:28.223204 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:28.223211 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:28.223239 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:05:28.223304 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:28.223311 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:28.223526 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:28.223663 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:28.224925 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:28.225928 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:28.227187 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:28.227733 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:28.227826 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:28.231835 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:28.231847 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:28.231862 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:28.231906 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:28.234535 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:28.235228 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:28.235235 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:28.235238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:28.235378 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:28.235675 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:28.235678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:28.235680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:28.236014 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:28.236298 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:28.236484 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 07:05:28.236683 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":255,"threads_started":1}
I20260812 07:05:28.238253 2719 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:05:28.318607 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 2425
I20260812 07:05:28.346131 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 2740
Aug 12 07:05:28 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.195@KRBTEST.COM
I20260812 07:05:28.361534 2993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:28.347552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34146 (local address 127.11.192.195:40087)
0812 07:05:28.348285 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:28.348292 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:28.348320 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:05:28.348380 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:28.348386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:28.348608 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:28.348734 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:28.349930 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:28.350942 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:28.352099 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:28.352685 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:28.352781 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:28.356790 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:28.356802 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:28.356815 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:28.356853 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:28.359228 (+ 2375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:28.359823 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:28.359829 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:28.359831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:28.359946 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:28.360262 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:28.360265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:28.360266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:28.360577 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:28.360818 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:28.361016 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 07:05:28.361245 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":346,"threads_started":1}
I20260812 07:05:28.362742 2972 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:05:28.442374 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 2740
I20260812 07:05:28.462685 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 1947
I20260812 07:05:28.463846 2054 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:05:28.567137 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 1947
I20260812 07:05:28.606956 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:35893
--webserver_interface=127.11.192.254
--webserver_port=34141
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:42129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:35893
--ranger_config_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.192.212:42425/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.192.212:42425/kms with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:05:28.924355 2998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:28.924620 2998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:28.924714 2998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:28.931631 2998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:05:28.931723 2998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:28.931775 2998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:28.931828 2998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:05:28.931874 2998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:05:28.941391 2998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:42129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.192.212:42425/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:35893
--ranger_config_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:35893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=34141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:28.942446 2998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:28.943638 2998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:05:28.954591 3013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:28.954641 3011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:28.954686 3008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:28.955587 2998 server_base.cc:1034] running on GCE node
I20260812 07:05:28.956314 2998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:28.957542 2998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:28.958729 2998 hybrid_clock.cc:648] HybridClock initialized: now 1786518328958680 us; error 41 us; skew 500 ppm
I20260812 07:05:28.959028 2998 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:28 dist-test-slave-lgr5 krb5kdc[1062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518328, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:28.961843 2998 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:05:28.963127 2998 webserver.cc:460] Webserver started at http://127.11.192.254:34141/ using document root <none> and password file <none>
I20260812 07:05:28.963704 2998 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:28.963829 2998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 07:05:29 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518328, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
I20260812 07:05:29.035106 2998 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:05:29.037972 3031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:29.039263 2998 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 07:05:29.039443 2998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "39531b6377634af38c86601f78b25fe9"
format_stamp: "Formatted at 2026-08-12 07:05:25 on dist-test-slave-lgr5"
server_key: "3ae9b3b77b4aa363bdc34648b41c1ab9"
server_key_iv: "94fa494caf77f00fc305d644780d210e"
server_key_version: "kuduclusterkey@0"
I20260812 07:05:29.040418 2998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:29.056110 2998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:29.059777 2998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:29.060034 2998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:29.076864 2998 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:35893
I20260812 07:05:29.076884 3157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:35893 every 8 connection(s)
I20260812 07:05:29.078249 2998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:05:29.081691 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 2998
I20260812 07:05:29.082746 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.192.212:42425/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:40479
--local_ip_for_outbound_sockets=127.11.192.193
--tserver_master_addrs=127.11.192.254:35893
--webserver_port=39129
--webserver_interface=127.11.192.193
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:42129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.192.212:42425/kms with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:05:29.086078 3158 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:05:29.090298 3158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9: Bootstrap starting.
I20260812 07:05:29.109407 3158 log.cc:824] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9: Log is configured to *not* fsync() on all Append() calls
I20260812 07:05:29.117965 3158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:05:29.118454 3158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9: Bootstrap complete.
I20260812 07:05:29.127059 3158 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39531b6377634af38c86601f78b25fe9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35893 } }
I20260812 07:05:29.127847 3158 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39531b6377634af38c86601f78b25fe9, State: Initialized, Role: FOLLOWER
I20260812 07:05:29.128456 3158 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39531b6377634af38c86601f78b25fe9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35893 } }
I20260812 07:05:29.128659 3158 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:05:29.128767 3158 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:05:29.128923 3158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [term 1 FOLLOWER]: Advancing to term 2
I20260812 07:05:29.132606 3158 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39531b6377634af38c86601f78b25fe9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35893 } }
I20260812 07:05:29.133042 3158 leader_election.cc:302] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39531b6377634af38c86601f78b25fe9; no voters:
I20260812 07:05:29.133592 3158 leader_election.cc:288] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 07:05:29.133771 3166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 07:05:29.134688 3166 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [term 2 LEADER]: Becoming Leader. State: Replica: 39531b6377634af38c86601f78b25fe9, State: Running, Role: LEADER
I20260812 07:05:29.135402 3158 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:05:29.135351 3166 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39531b6377634af38c86601f78b25fe9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35893 } }
I20260812 07:05:29.144766 3167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "39531b6377634af38c86601f78b25fe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39531b6377634af38c86601f78b25fe9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35893 } } }
I20260812 07:05:29.145143 3167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [sys.catalog]: This master's current role is: LEADER
I20260812 07:05:29.145664 3168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39531b6377634af38c86601f78b25fe9. Latest consensus state: current_term: 2 leader_uuid: "39531b6377634af38c86601f78b25fe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39531b6377634af38c86601f78b25fe9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35893 } } }
I20260812 07:05:29.145928 3168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9 [sys.catalog]: This master's current role is: LEADER
I20260812 07:05:29.153224 3188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:05:29.153748 3158 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 07:05:29.172739 3188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:05:29.174647 3188 catalog_manager.cc:1259] Loaded cluster ID: 57e39b20137c4dca8667115108524cf0
I20260812 07:05:29.174899 3188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:05:29.179337 3188 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:05:29.181102 3188 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9: Loaded TSK: 0
I20260812 07:05:29.182756 3188 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 07:05:29.525233 3163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:29.525648 3163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:29.525832 3163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:29.536885 3163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:29.537083 3163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:29.537276 3163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:05:29.552049 3163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:42129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.192.212:42425/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:40479
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=39129
--tserver_master_addrs=127.11.192.254:35893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:29.553661 3163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:29.555438 3163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:05:29.573415 3223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:29.576502 3224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:29.577950 3230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:29.578392 3163 server_base.cc:1034] running on GCE node
I20260812 07:05:29.579257 3163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:29.580521 3163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:29.581779 3163 hybrid_clock.cc:648] HybridClock initialized: now 1786518329581715 us; error 48 us; skew 500 ppm
I20260812 07:05:29.582155 3163 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:29 dist-test-slave-lgr5 krb5kdc[1062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518329, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:29.587059 3163 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:05:29.588739 3163 webserver.cc:460] Webserver started at http://127.11.192.193:39129/ using document root <none> and password file <none>
I20260812 07:05:29.589691 3163 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:29.589874 3163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 07:05:29 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518329, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
I20260812 07:05:29.623798 3163 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.004s
I20260812 07:05:29.633304 3244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:29.635237 3163 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.005s sys 0.004s
I20260812 07:05:29.635491 3163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "5935e032929a495da3d36d526b0ab0ed"
format_stamp: "Formatted at 2026-08-12 07:05:26 on dist-test-slave-lgr5"
server_key: "f7646cb30619358f919a7ad717382235"
server_key_iv: "5bd9be268866e5e3e753f1498625f00a"
server_key_version: "kuduclusterkey@0"
I20260812 07:05:29.637030 3163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:29.692710 3163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:29.698071 3163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:29.698385 3163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:29.700359 3163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:05:29.702222 3163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:05:29.702330 3163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:29.702436 3163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:05:29.702508 3163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:29.787132 3163 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:40479
I20260812 07:05:29.787197 3468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:40479 every 8 connection(s)
I20260812 07:05:29.788609 3163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:05:29.795199 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 3163
I20260812 07:05:29.796113 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.192.212:42425/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:36327
--local_ip_for_outbound_sockets=127.11.192.194
--tserver_master_addrs=127.11.192.254:35893
--webserver_port=37383
--webserver_interface=127.11.192.194
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:42129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.192.212:42425/kms with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:05:29 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518329, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:05:29.851491 3472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:29.793296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:35893 (local address 127.11.192.193:38085)
0812 07:05:29.811510 (+ 18214us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:29.811582 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:29.815191 (+ 3609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:29.819524 (+ 4333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:29.819537 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:29.820266 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:05:29.821147 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:29.821174 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:29.824592 (+ 3418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:29.824597 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:29.826380 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:29.826396 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:29.831121 (+ 4725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:29.831131 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:29.831427 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:29.832289 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:05:29.832321 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:05:29.835148 (+ 2827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:05:29.846225 (+ 11077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:29.846234 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:29.846249 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:29.846845 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:29.847471 (+ 626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:29.847476 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:29.847480 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:29.847749 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:29.848811 (+ 1062us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:05:29.848820 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:05:29.849235 (+ 415us) client_negotiation.cc:769] Sending connection context
0812 07:05:29.849532 (+ 297us) client_negotiation.cc:240] Negotiation successful
0812 07:05:29.850090 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17812,"thread_start_us":18653,"threads_started":1}
I20260812 07:05:29.862226 3473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:29.793988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:38085 (local address 127.11.192.254:35893)
0812 07:05:29.817563 (+ 23575us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:29.817584 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:29.817628 (+ 44us) server_negotiation.cc:407] Connection header received
0812 07:05:29.818081 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:29.818101 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:29.818588 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:29.821442 (+ 2854us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:29.821898 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:29.823771 (+ 1873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:29.826573 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:29.830827 (+ 4254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:29.839423 (+ 8596us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:29.839783 (+ 360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:29.839795 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:29.839814 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:29.839864 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:29.846054 (+ 6190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:29.847060 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:29.847069 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:29.847078 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:29.847325 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:29.847906 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:29.847910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:29.847913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:29.848633 (+ 720us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:29.860246 (+ 11613us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:29.860526 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 07:05:29.860967 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23283,"thread_start_us":24867,"threads_started":1}
I20260812 07:05:29.865639 3469 heartbeater.cc:347] Connected to a master server at 127.11.192.254:35893
I20260812 07:05:29.865993 3469 heartbeater.cc:464] Registering TS with master...
I20260812 07:05:29.866926 3469 heartbeater.cc:511] Master 127.11.192.254:35893 requested a full tablet report, sending...
W20260812 07:05:29.870173 3469 heartbeater.cc:644] Failed to heartbeat to 127.11.192.254:35893 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 07:05:29.870402 3469 heartbeater.cc:464] Registering TS with master...
I20260812 07:05:29.870682 3469 heartbeater.cc:511] Master 127.11.192.254:35893 requested a full tablet report, sending...
I20260812 07:05:29.872018 3469 heartbeater.cc:464] Registering TS with master...
I20260812 07:05:29.872323 3469 heartbeater.cc:511] Master 127.11.192.254:35893 requested a full tablet report, sending...
W20260812 07:05:29.873556 3469 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 07:05:30.248358 3476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:30.248631 3476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:30.248795 3476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:30.259934 3476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:30.260142 3476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:30.260370 3476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:05:30.275714 3476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:42129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.192.212:42425/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:36327
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=37383
--tserver_master_addrs=127.11.192.254:35893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:30.277519 3476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:30.279264 3476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:05:30.296020 3516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:30.298321 3518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:30.305181 3476 server_base.cc:1034] running on GCE node
W20260812 07:05:30.305473 3521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:30.306329 3476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:30.307530 3476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:30.308807 3476 hybrid_clock.cc:648] HybridClock initialized: now 1786518330308733 us; error 61 us; skew 500 ppm
I20260812 07:05:30.309219 3476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:30 dist-test-slave-lgr5 krb5kdc[1062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518330, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:30.320601 3476 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:05:30.322669 3476 webserver.cc:460] Webserver started at http://127.11.192.194:37383/ using document root <none> and password file <none>
I20260812 07:05:30.323654 3476 fs_manager.cc:359] Metadata directory not provided
Aug 12 07:05:30 dist-test-slave-lgr5 krb5kdc[1062](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786518330, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:30.323808 3476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 07:05:30 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518330, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
I20260812 07:05:30.365134 3476 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.002s sys 0.001s
I20260812 07:05:30.370633 3539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:30.372447 3476 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:05:30.372658 3476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "45e457a2ac3947068168483529c63743"
format_stamp: "Formatted at 2026-08-12 07:05:27 on dist-test-slave-lgr5"
server_key: "bba5e56a2ab0a383ec3228ea7ddd3291"
server_key_iv: "feb008780210150335d4d622637d3b60"
server_key_version: "kuduclusterkey@0"
I20260812 07:05:30.374109 3476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:30.391610 3476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:30.396647 3476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:30.396976 3476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:30.398222 3476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:05:30.400022 3476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:05:30.400215 3476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260812 07:05:30.400343 3476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:05:30.400424 3476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:30.467264 3476 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:36327
I20260812 07:05:30.467356 3757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:36327 every 8 connection(s)
I20260812 07:05:30.469244 3476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:05:30.470114 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 3476
I20260812 07:05:30.471028 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.192.212:42425/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:40087
--local_ip_for_outbound_sockets=127.11.192.195
--tserver_master_addrs=127.11.192.254:35893
--webserver_port=33195
--webserver_interface=127.11.192.195
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:42129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.192.212:42425/kms with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:05:30 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518330, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:05:30.511746 3762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:30.472293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:41989 (local address 127.11.192.254:35893)
0812 07:05:30.482936 (+ 10643us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:30.482944 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:30.485366 (+ 2422us) server_negotiation.cc:407] Connection header received
0812 07:05:30.487281 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:30.487286 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:30.487378 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:30.487479 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:30.491615 (+ 4136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:30.493882 (+ 2267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:30.498991 (+ 5109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:30.499446 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:30.499582 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:30.503006 (+ 3424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:30.503011 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:30.503015 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:30.503053 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:30.506593 (+ 3540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:30.507608 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:30.507615 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:30.507618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:30.507787 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:30.508596 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:30.508601 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:30.508605 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:30.509010 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:30.509184 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:30.511035 (+ 1851us) server_negotiation.cc:299] Negotiation successful
0812 07:05:30.511275 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10503,"thread_start_us":10397,"threads_started":1}
I20260812 07:05:30.513916 3766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:30.472908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:35893 (local address 127.11.192.194:41989)
0812 07:05:30.484119 (+ 11211us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:30.484188 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:30.486929 (+ 2741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:30.488589 (+ 1660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:30.488602 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:30.489320 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:05:30.491357 (+ 2037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:30.491387 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:30.494108 (+ 2721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:30.494114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:30.498826 (+ 4712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:30.498841 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:30.499706 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:30.499714 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:30.499944 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:30.500643 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:05:30.500668 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:05:30.502751 (+ 2083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:05:30.506784 (+ 4033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:30.506794 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:30.506810 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:30.507437 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:30.507949 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:30.507955 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:30.507959 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:30.508427 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:30.509269 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:05:30.509278 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:05:30.509861 (+ 583us) client_negotiation.cc:769] Sending connection context
0812 07:05:30.510204 (+ 343us) client_negotiation.cc:240] Negotiation successful
0812 07:05:30.512473 (+ 2269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10857,"thread_start_us":10594,"threads_started":1}
I20260812 07:05:30.516012 3759 heartbeater.cc:347] Connected to a master server at 127.11.192.254:35893
I20260812 07:05:30.516396 3759 heartbeater.cc:464] Registering TS with master...
I20260812 07:05:30.517288 3759 heartbeater.cc:511] Master 127.11.192.254:35893 requested a full tablet report, sending...
I20260812 07:05:30.519536 3076 ts_manager.cc:194] Registered new tserver with Master: 45e457a2ac3947068168483529c63743 (127.11.192.194:36327)
I20260812 07:05:30.521755 3076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:41989
Aug 12 07:05:30 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518330, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
I20260812 07:05:30.873726 3469 heartbeater.cc:464] Registering TS with master...
I20260812 07:05:30.874195 3469 heartbeater.cc:511] Master 127.11.192.254:35893 requested a full tablet report, sending...
I20260812 07:05:30.875682 3076 ts_manager.cc:194] Registered new tserver with Master: 5935e032929a495da3d36d526b0ab0ed (127.11.192.193:40479)
I20260812 07:05:30.876927 3076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:38085
W20260812 07:05:30.925947 3768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:30.926404 3768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:30.926611 3768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:30.937978 3768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:30.938185 3768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:30.938438 3768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:05:30.953343 3768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:42129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.192.212:42425/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:40087
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=33195
--tserver_master_addrs=127.11.192.254:35893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:30.955229 3768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:30.957005 3768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:05:30.973112 3783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:30.974321 3784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:30.982206 3793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:30.982632 3768 server_base.cc:1034] running on GCE node
I20260812 07:05:30.983239 3768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:30.984290 3768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:30.985551 3768 hybrid_clock.cc:648] HybridClock initialized: now 1786518330985491 us; error 53 us; skew 500 ppm
I20260812 07:05:30.985875 3768 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:30 dist-test-slave-lgr5 krb5kdc[1062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518330, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:30.989436 3768 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:05:31.001832 3768 webserver.cc:460] Webserver started at http://127.11.192.195:33195/ using document root <none> and password file <none>
I20260812 07:05:31.002820 3768 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:31.003001 3768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 07:05:31 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518330, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
I20260812 07:05:31.031590 3768 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.006s sys 0.004s
I20260812 07:05:31.046725 3808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:31.048324 3768 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.008s sys 0.006s
I20260812 07:05:31.048588 3768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "65a743ddefbc40fbb298aec04d085a7e"
format_stamp: "Formatted at 2026-08-12 07:05:27 on dist-test-slave-lgr5"
server_key: "717aaf65fb3123529663080df966badc"
server_key_iv: "6e3ea5073a1204ef8b4809bb30960fa2"
server_key_version: "kuduclusterkey@0"
I20260812 07:05:31.049872 3768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 12 07:05:31 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518330, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for HTTP/127.11.192.212@KRBTEST.COM
I20260812 07:05:31.102680 3768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:31.108137 3768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:31.108531 3768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:31.126802 3768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:05:31.154537 3768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:05:31.154752 3768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:31.154925 3768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:05:31.155061 3768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:31.306921 3768 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:40087
I20260812 07:05:31.306982 4023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:40087 every 8 connection(s)
I20260812 07:05:31.309587 3768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:05:31.310688 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 3768
I20260812 07:05:31.333747 4034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:31.311842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47562 (local address 127.11.192.254:35893)
0812 07:05:31.312935 (+ 1093us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:31.312941 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:31.315842 (+ 2901us) server_negotiation.cc:407] Connection header received
0812 07:05:31.316445 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:31.316451 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:31.316544 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:31.316668 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:31.318707 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.319955 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.322615 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.323315 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.323443 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:31.327479 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:31.327485 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:31.327489 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:31.327532 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:31.330678 (+ 3146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:31.331620 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:31.331625 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:31.331629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:31.331788 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:31.332285 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:31.332291 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:31.332295 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:31.332588 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:31.332903 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:31.333186 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 07:05:31.333395 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":977,"thread_start_us":856,"threads_started":1}
Aug 12 07:05:31 dist-test-slave-lgr5 krb5kdc[1062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518330, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:05:31.342437 4042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:31.320780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:59449 (local address 127.11.192.254:35893)
0812 07:05:31.322782 (+ 2002us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:31.322789 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:31.324983 (+ 2194us) server_negotiation.cc:407] Connection header received
0812 07:05:31.327032 (+ 2049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:31.327037 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:31.327124 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:31.327212 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:31.329117 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.330598 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.332363 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.333167 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.333301 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:31.337214 (+ 3913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:31.337219 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:31.337223 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:31.337259 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:31.339580 (+ 2321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:31.340268 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:31.340274 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:31.340277 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:31.340386 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:31.340819 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:31.340825 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:31.340841 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:31.341081 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:31.341237 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:31.341903 (+ 666us) server_negotiation.cc:299] Negotiation successful
0812 07:05:31.342096 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1850,"thread_start_us":1680,"threads_started":1}
I20260812 07:05:31.343302 4043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:31.321289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:35893 (local address 127.11.192.195:59449)
0812 07:05:31.324836 (+ 3547us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:31.324880 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:31.326762 (+ 1882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:31.327500 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:31.327512 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:31.328120 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:05:31.328863 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:31.328900 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.330770 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.330775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:31.332210 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:31.332224 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.333641 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.333650 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:31.333960 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:31.334653 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:05:31.334680 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:05:31.336994 (+ 2314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:05:31.339706 (+ 2712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:31.339712 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:31.339722 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:31.340154 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:31.340484 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:31.340488 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:31.340492 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:31.340711 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:31.341203 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:05:31.341210 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:05:31.341540 (+ 330us) client_negotiation.cc:769] Sending connection context
0812 07:05:31.341753 (+ 213us) client_negotiation.cc:240] Negotiation successful
0812 07:05:31.342261 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3274,"thread_start_us":3065,"threads_started":1}
I20260812 07:05:31.344537 4025 heartbeater.cc:347] Connected to a master server at 127.11.192.254:35893
I20260812 07:05:31.344794 4025 heartbeater.cc:464] Registering TS with master...
I20260812 07:05:31.345436 4025 heartbeater.cc:511] Master 127.11.192.254:35893 requested a full tablet report, sending...
I20260812 07:05:31.346737 3076 ts_manager.cc:194] Registered new tserver with Master: 65a743ddefbc40fbb298aec04d085a7e (127.11.192.195:40087)
I20260812 07:05:31.347671 3076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:59449
I20260812 07:05:31.348758 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:05:31.363742 4042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:31.351448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47566 (local address 127.11.192.254:35893)
0812 07:05:31.351575 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:31.351579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:31.352031 (+ 452us) server_negotiation.cc:407] Connection header received
0812 07:05:31.352202 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:31.352206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:31.352259 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:31.352331 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:31.353673 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.354384 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.356011 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.356729 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.356851 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:31.359526 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:31.359531 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:31.359534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:31.359562 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:31.361742 (+ 2180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:31.362357 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:31.362360 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:31.362362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:31.362464 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:31.362739 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:31.362741 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:31.362743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:31.362977 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:31.363141 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:31.363384 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 07:05:31.363538 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 07:05:31.367813 3076 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47566:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:05:31.375020 3076 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:05:31.411427 4080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:31.389290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:40087 (local address 127.0.0.1:59396)
0812 07:05:31.403200 (+ 13910us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:31.403217 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:31.403400 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:31.404311 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:31.404315 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:31.404339 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:05:31.404715 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:31.404729 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.406551 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.406556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:31.408552 (+ 1996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:31.408565 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.410419 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.410425 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:31.410635 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:31.410688 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 07:05:31.410794 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 07:05:31.410979 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13796,"mutex_wait_us":27,"thread_start_us":13030,"threads_started":1}
I20260812 07:05:31.412047 4081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:31.388635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50244 (local address 127.11.192.193:40479)
0812 07:05:31.402958 (+ 14323us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:31.402966 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:31.402992 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:05:31.403095 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:31.403103 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:31.403353 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:31.403561 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:05:31.404518 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.406175 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.409272 (+ 3097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.410646 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.410850 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:31.411020 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:31.411444 (+ 424us) server_negotiation.cc:299] Negotiation successful
0812 07:05:31.411776 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14146,"thread_start_us":13976,"threads_started":1}
I20260812 07:05:31.412176 4085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:31.389744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59396 (local address 127.11.192.195:40087)
0812 07:05:31.401930 (+ 12186us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:31.401939 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:31.403306 (+ 1367us) server_negotiation.cc:407] Connection header received
0812 07:05:31.403575 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:31.403582 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:31.403788 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:31.403939 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:05:31.404897 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.406401 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.408699 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.409885 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.410070 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:31.410252 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:31.411685 (+ 1433us) server_negotiation.cc:299] Negotiation successful
0812 07:05:31.411840 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12000,"thread_start_us":12192,"threads_started":1}
I20260812 07:05:31.411757 4083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:31.388284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:40479 (local address 127.0.0.1:50244)
0812 07:05:31.402568 (+ 14284us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:31.402587 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:31.402703 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:31.403846 (+ 1143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:31.403853 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:31.403883 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:05:31.404299 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:31.404311 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.406696 (+ 2385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.406701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:31.409107 (+ 2406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:31.409123 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.410881 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.410888 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:31.411059 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:31.411118 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 07:05:31.411189 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 07:05:31.411311 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14169,"mutex_wait_us":40,"thread_start_us":14610,"threads_started":1}
I20260812 07:05:31.411917 4077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:31.387887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:36327 (local address 127.0.0.1:45290)
0812 07:05:31.388932 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:31.388986 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:31.389145 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:31.405270 (+ 16125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:31.405274 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:31.405297 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:05:31.405582 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:31.405591 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.407321 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.407326 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:31.409469 (+ 2143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:31.409484 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.411066 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.411072 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:31.411296 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:31.411328 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 07:05:31.411431 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 07:05:31.411552 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":803,"mutex_wait_us":63,"thread_start_us":664,"threads_started":1}
I20260812 07:05:31.412518 4078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:31.388110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45290 (local address 127.11.192.194:36327)
0812 07:05:31.404844 (+ 16734us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:31.404852 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:31.404875 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:05:31.404948 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:31.404953 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:31.405103 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:31.405257 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:05:31.405747 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.407184 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.409646 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.410818 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.411165 (+ 347us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:31.411318 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:31.412053 (+ 735us) server_negotiation.cc:299] Negotiation successful
0812 07:05:31.412247 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16573,"thread_start_us":16466,"threads_started":1}
I20260812 07:05:31.415757 3903 tablet_service.cc:1467] Processing CreateTablet for tablet 35cbf50e0a76451289797386041ca0eb (DEFAULT_TABLE table=test-table [id=749f4530633443278662cb62c3e600ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:05:31.415756 3350 tablet_service.cc:1467] Processing CreateTablet for tablet 35cbf50e0a76451289797386041ca0eb (DEFAULT_TABLE table=test-table [id=749f4530633443278662cb62c3e600ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:05:31.416983 3903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35cbf50e0a76451289797386041ca0eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:05:31.417126 3350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35cbf50e0a76451289797386041ca0eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:05:31.418510 3635 tablet_service.cc:1467] Processing CreateTablet for tablet 35cbf50e0a76451289797386041ca0eb (DEFAULT_TABLE table=test-table [id=749f4530633443278662cb62c3e600ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:05:31.419729 3635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35cbf50e0a76451289797386041ca0eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:05:31.428550 4086 tablet_bootstrap.cc:492] T 35cbf50e0a76451289797386041ca0eb P 65a743ddefbc40fbb298aec04d085a7e: Bootstrap starting.
I20260812 07:05:31.431711 4087 tablet_bootstrap.cc:492] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed: Bootstrap starting.
I20260812 07:05:31.432807 4086 tablet_bootstrap.cc:654] T 35cbf50e0a76451289797386041ca0eb P 65a743ddefbc40fbb298aec04d085a7e: Neither blocks nor log segments found. Creating new log.
I20260812 07:05:31.434093 4089 tablet_bootstrap.cc:492] T 35cbf50e0a76451289797386041ca0eb P 45e457a2ac3947068168483529c63743: Bootstrap starting.
I20260812 07:05:31.434082 4086 log.cc:824] T 35cbf50e0a76451289797386041ca0eb P 65a743ddefbc40fbb298aec04d085a7e: Log is configured to *not* fsync() on all Append() calls
I20260812 07:05:31.435266 4087 tablet_bootstrap.cc:654] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed: Neither blocks nor log segments found. Creating new log.
I20260812 07:05:31.436604 4087 log.cc:824] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed: Log is configured to *not* fsync() on all Append() calls
I20260812 07:05:31.437086 4089 tablet_bootstrap.cc:654] T 35cbf50e0a76451289797386041ca0eb P 45e457a2ac3947068168483529c63743: Neither blocks nor log segments found. Creating new log.
I20260812 07:05:31.438264 4089 log.cc:824] T 35cbf50e0a76451289797386041ca0eb P 45e457a2ac3947068168483529c63743: Log is configured to *not* fsync() on all Append() calls
I20260812 07:05:31.439592 4086 tablet_bootstrap.cc:492] T 35cbf50e0a76451289797386041ca0eb P 65a743ddefbc40fbb298aec04d085a7e: No bootstrap required, opened a new log
I20260812 07:05:31.439613 4087 tablet_bootstrap.cc:492] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed: No bootstrap required, opened a new log
I20260812 07:05:31.439864 4086 ts_tablet_manager.cc:1397] T 35cbf50e0a76451289797386041ca0eb P 65a743ddefbc40fbb298aec04d085a7e: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260812 07:05:31.439877 4087 ts_tablet_manager.cc:1397] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260812 07:05:31.441126 4089 tablet_bootstrap.cc:492] T 35cbf50e0a76451289797386041ca0eb P 45e457a2ac3947068168483529c63743: No bootstrap required, opened a new log
I20260812 07:05:31.441362 4089 ts_tablet_manager.cc:1397] T 35cbf50e0a76451289797386041ca0eb P 45e457a2ac3947068168483529c63743: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 07:05:31.447485 4087 raft_consensus.cc:348] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e457a2ac3947068168483529c63743" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36327 } } peers { permanent_uuid: "65a743ddefbc40fbb298aec04d085a7e" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 40087 } } peers { permanent_uuid: "5935e032929a495da3d36d526b0ab0ed" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40479 } }
I20260812 07:05:31.447803 4087 raft_consensus.cc:374] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:05:31.447898 4087 raft_consensus.cc:729] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5935e032929a495da3d36d526b0ab0ed, State: Initialized, Role: FOLLOWER
I20260812 07:05:31.448482 4087 consensus_queue.cc:260] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e457a2ac3947068168483529c63743" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36327 } } peers { permanent_uuid: "65a743ddefbc40fbb298aec04d085a7e" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 40087 } } peers { permanent_uuid: "5935e032929a495da3d36d526b0ab0ed" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40479 } }
I20260812 07:05:31.449496 3469 heartbeater.cc:503] Master 127.11.192.254:35893 was elected leader, sending a full tablet report...
I20260812 07:05:31.450115 4087 ts_tablet_manager.cc:1428] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260812 07:05:31.450373 4086 raft_consensus.cc:348] T 35cbf50e0a76451289797386041ca0eb P 65a743ddefbc40fbb298aec04d085a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e457a2ac3947068168483529c63743" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36327 } } peers { permanent_uuid: "65a743ddefbc40fbb298aec04d085a7e" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 40087 } } peers { permanent_uuid: "5935e032929a495da3d36d526b0ab0ed" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40479 } }
I20260812 07:05:31.450729 4086 raft_consensus.cc:374] T 35cbf50e0a76451289797386041ca0eb P 65a743ddefbc40fbb298aec04d085a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:05:31.450841 4086 raft_consensus.cc:729] T 35cbf50e0a76451289797386041ca0eb P 65a743ddefbc40fbb298aec04d085a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65a743ddefbc40fbb298aec04d085a7e, State: Initialized, Role: FOLLOWER
I20260812 07:05:31.451203 4089 raft_consensus.cc:348] T 35cbf50e0a76451289797386041ca0eb P 45e457a2ac3947068168483529c63743 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e457a2ac3947068168483529c63743" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36327 } } peers { permanent_uuid: "65a743ddefbc40fbb298aec04d085a7e" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 40087 } } peers { permanent_uuid: "5935e032929a495da3d36d526b0ab0ed" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40479 } }
I20260812 07:05:31.451401 4086 consensus_queue.cc:260] T 35cbf50e0a76451289797386041ca0eb P 65a743ddefbc40fbb298aec04d085a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e457a2ac3947068168483529c63743" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36327 } } peers { permanent_uuid: "65a743ddefbc40fbb298aec04d085a7e" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 40087 } } peers { permanent_uuid: "5935e032929a495da3d36d526b0ab0ed" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40479 } }
I20260812 07:05:31.451586 4089 raft_consensus.cc:374] T 35cbf50e0a76451289797386041ca0eb P 45e457a2ac3947068168483529c63743 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:05:31.451668 4089 raft_consensus.cc:729] T 35cbf50e0a76451289797386041ca0eb P 45e457a2ac3947068168483529c63743 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e457a2ac3947068168483529c63743, State: Initialized, Role: FOLLOWER
I20260812 07:05:31.452590 4025 heartbeater.cc:503] Master 127.11.192.254:35893 was elected leader, sending a full tablet report...
I20260812 07:05:31.452322 4089 consensus_queue.cc:260] T 35cbf50e0a76451289797386041ca0eb P 45e457a2ac3947068168483529c63743 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e457a2ac3947068168483529c63743" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36327 } } peers { permanent_uuid: "65a743ddefbc40fbb298aec04d085a7e" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 40087 } } peers { permanent_uuid: "5935e032929a495da3d36d526b0ab0ed" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40479 } }
I20260812 07:05:31.453140 4086 ts_tablet_manager.cc:1428] T 35cbf50e0a76451289797386041ca0eb P 65a743ddefbc40fbb298aec04d085a7e: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260812 07:05:31.456249 3759 heartbeater.cc:503] Master 127.11.192.254:35893 was elected leader, sending a full tablet report...
I20260812 07:05:31.456799 4089 ts_tablet_manager.cc:1428] T 35cbf50e0a76451289797386041ca0eb P 45e457a2ac3947068168483529c63743: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
W20260812 07:05:31.484275 3760 tablet.cc:2369] T 35cbf50e0a76451289797386041ca0eb P 45e457a2ac3947068168483529c63743: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:05:31.569556 4038 tablet.cc:2369] T 35cbf50e0a76451289797386041ca0eb P 65a743ddefbc40fbb298aec04d085a7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:05:31.575907 3470 tablet.cc:2369] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:05:31.656186 4093 raft_consensus.cc:482] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:05:31.656488 4093 raft_consensus.cc:504] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e457a2ac3947068168483529c63743" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36327 } } peers { permanent_uuid: "65a743ddefbc40fbb298aec04d085a7e" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 40087 } } peers { permanent_uuid: "5935e032929a495da3d36d526b0ab0ed" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40479 } }
I20260812 07:05:31.658188 4093 leader_election.cc:288] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e457a2ac3947068168483529c63743 (127.11.192.194:36327), 65a743ddefbc40fbb298aec04d085a7e (127.11.192.195:40087)
I20260812 07:05:31.665055 4097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:31.658415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:36327 (local address 127.11.192.193:41855)
0812 07:05:31.659066 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:31.659085 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:31.659220 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:31.659776 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:31.659781 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:31.659822 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:05:31.660218 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:31.660232 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.661425 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.661431 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:31.663021 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:31.663037 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.664341 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.664347 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:31.664524 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:31.664561 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 07:05:31.664646 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 07:05:31.664738 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":322,"threads_started":1}
I20260812 07:05:31.665074 4078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:31.658621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:41855 (local address 127.11.192.194:36327)
0812 07:05:31.658803 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:31.658807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:31.659155 (+ 348us) server_negotiation.cc:407] Connection header received
0812 07:05:31.659398 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:31.659404 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:31.659490 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:31.659587 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:05:31.660371 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.661283 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.663178 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.664142 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.664255 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:31.664306 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:31.664691 (+ 385us) server_negotiation.cc:299] Negotiation successful
0812 07:05:31.664814 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:05:31.665244 4085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:31.659439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:53803 (local address 127.11.192.195:40087)
0812 07:05:31.659671 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:31.659677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:31.660042 (+ 365us) server_negotiation.cc:407] Connection header received
0812 07:05:31.660217 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:31.660221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:31.660298 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:31.660381 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:05:31.660919 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.661753 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.663522 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.664363 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.664509 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:31.664576 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:31.664899 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 07:05:31.665030 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 07:05:31.665674 4102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:31.659205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:40087 (local address 127.11.192.193:53803)
0812 07:05:31.659935 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:31.659952 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:31.660067 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:31.660408 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:31.660412 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:31.660438 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:05:31.660792 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:31.660805 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.661898 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.661902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:31.663367 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:31.663382 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.664565 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.664570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:31.664727 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:31.664760 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:05:31.664859 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 07:05:31.664989 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":426,"threads_started":1}
I20260812 07:05:31.666203 3668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35cbf50e0a76451289797386041ca0eb" candidate_uuid: "5935e032929a495da3d36d526b0ab0ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e457a2ac3947068168483529c63743" is_pre_election: true
I20260812 07:05:31.666546 3949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35cbf50e0a76451289797386041ca0eb" candidate_uuid: "5935e032929a495da3d36d526b0ab0ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65a743ddefbc40fbb298aec04d085a7e" is_pre_election: true
I20260812 07:05:31.666671 3668 raft_consensus.cc:2454] T 35cbf50e0a76451289797386041ca0eb P 45e457a2ac3947068168483529c63743 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5935e032929a495da3d36d526b0ab0ed in term 0.
I20260812 07:05:31.666956 3949 raft_consensus.cc:2454] T 35cbf50e0a76451289797386041ca0eb P 65a743ddefbc40fbb298aec04d085a7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5935e032929a495da3d36d526b0ab0ed in term 0.
I20260812 07:05:31.667266 3251 leader_election.cc:302] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e457a2ac3947068168483529c63743, 5935e032929a495da3d36d526b0ab0ed; no voters:
I20260812 07:05:31.667591 4093 raft_consensus.cc:2781] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:05:31.667732 4093 raft_consensus.cc:482] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:05:31.667821 4093 raft_consensus.cc:3037] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:05:31.669973 4093 raft_consensus.cc:504] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e457a2ac3947068168483529c63743" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36327 } } peers { permanent_uuid: "65a743ddefbc40fbb298aec04d085a7e" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 40087 } } peers { permanent_uuid: "5935e032929a495da3d36d526b0ab0ed" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40479 } }
I20260812 07:05:31.670478 4093 leader_election.cc:288] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed [CANDIDATE]: Term 1 election: Requested vote from peers 45e457a2ac3947068168483529c63743 (127.11.192.194:36327), 65a743ddefbc40fbb298aec04d085a7e (127.11.192.195:40087)
I20260812 07:05:31.670878 3668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35cbf50e0a76451289797386041ca0eb" candidate_uuid: "5935e032929a495da3d36d526b0ab0ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e457a2ac3947068168483529c63743"
I20260812 07:05:31.670934 3949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35cbf50e0a76451289797386041ca0eb" candidate_uuid: "5935e032929a495da3d36d526b0ab0ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65a743ddefbc40fbb298aec04d085a7e"
I20260812 07:05:31.671098 3668 raft_consensus.cc:3037] T 35cbf50e0a76451289797386041ca0eb P 45e457a2ac3947068168483529c63743 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:05:31.671115 3949 raft_consensus.cc:3037] T 35cbf50e0a76451289797386041ca0eb P 65a743ddefbc40fbb298aec04d085a7e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:05:31.674151 3668 raft_consensus.cc:2454] T 35cbf50e0a76451289797386041ca0eb P 45e457a2ac3947068168483529c63743 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5935e032929a495da3d36d526b0ab0ed in term 1.
I20260812 07:05:31.674151 3949 raft_consensus.cc:2454] T 35cbf50e0a76451289797386041ca0eb P 65a743ddefbc40fbb298aec04d085a7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5935e032929a495da3d36d526b0ab0ed in term 1.
I20260812 07:05:31.674530 3251 leader_election.cc:302] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e457a2ac3947068168483529c63743, 5935e032929a495da3d36d526b0ab0ed; no voters:
I20260812 07:05:31.674762 4093 raft_consensus.cc:2781] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:05:31.675161 4093 raft_consensus.cc:686] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed [term 1 LEADER]: Becoming Leader. State: Replica: 5935e032929a495da3d36d526b0ab0ed, State: Running, Role: LEADER
I20260812 07:05:31.675583 4093 consensus_queue.cc:237] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e457a2ac3947068168483529c63743" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36327 } } peers { permanent_uuid: "65a743ddefbc40fbb298aec04d085a7e" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 40087 } } peers { permanent_uuid: "5935e032929a495da3d36d526b0ab0ed" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40479 } }
I20260812 07:05:31.679739 3076 catalog_manager.cc:5495] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 5935e032929a495da3d36d526b0ab0ed (127.11.192.193). New cstate: current_term: 1 leader_uuid: "5935e032929a495da3d36d526b0ab0ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e457a2ac3947068168483529c63743" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65a743ddefbc40fbb298aec04d085a7e" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 40087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5935e032929a495da3d36d526b0ab0ed" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40479 } health_report { overall_health: HEALTHY } } }
I20260812 07:05:31.701998 4081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:31.696558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50256 (local address 127.11.192.193:40479)
0812 07:05:31.696751 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:31.696757 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:31.696842 (+ 85us) server_negotiation.cc:407] Connection header received
0812 07:05:31.697080 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:31.697085 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:31.697166 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:31.697288 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:05:31.697665 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.698711 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.700027 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.700456 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.700545 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:31.701048 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:05:31.701515 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:05:31.701620 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:31.701693 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 07:05:31.701755 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:05:31.709856 3668 raft_consensus.cc:1261] T 35cbf50e0a76451289797386041ca0eb P 45e457a2ac3947068168483529c63743 [term 1 FOLLOWER]: Refusing update from remote peer 5935e032929a495da3d36d526b0ab0ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:05:31.709856 3949 raft_consensus.cc:1261] T 35cbf50e0a76451289797386041ca0eb P 65a743ddefbc40fbb298aec04d085a7e [term 1 FOLLOWER]: Refusing update from remote peer 5935e032929a495da3d36d526b0ab0ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:05:31.710738 4105 consensus_queue.cc:1035] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "65a743ddefbc40fbb298aec04d085a7e" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 40087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:05:31.711066 4093 consensus_queue.cc:1035] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e457a2ac3947068168483529c63743" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 36327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:05:31.741873 3074 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47566:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:05:31.742219 3074 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47566:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:05:31.746169 3074 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 39531b6377634af38c86601f78b25fe9: Sending DeleteTablet for 3 replicas of tablet 35cbf50e0a76451289797386041ca0eb
I20260812 07:05:31.747157 3635 tablet_service.cc:1514] Processing DeleteTablet for tablet 35cbf50e0a76451289797386041ca0eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:05:31 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:45290
I20260812 07:05:31.747321 3903 tablet_service.cc:1514] Processing DeleteTablet for tablet 35cbf50e0a76451289797386041ca0eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:05:31 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:59396
I20260812 07:05:31.747332 3350 tablet_service.cc:1514] Processing DeleteTablet for tablet 35cbf50e0a76451289797386041ca0eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:05:31 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:50244
I20260812 07:05:31.755710 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 3163
I20260812 07:05:31.765013 4126 tablet_replica.cc:330] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed: stopping tablet replica
I20260812 07:05:31.768841 4126 raft_consensus.cc:2229] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:05:31.769639 4126 raft_consensus.cc:2258] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:05:31.772521 4126 ts_tablet_manager.cc:1905] T 35cbf50e0a76451289797386041ca0eb P 5935e032929a495da3d36d526b0ab0ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:05:31.775261 4124 tablet_replica.cc:330] T 35cbf50e0a76451289797386041ca0eb P 45e457a2ac3947068168483529c63743: stopping tablet replica
I20260812 07:05:31.776620 4081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:31.758971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50260 (local address 127.11.192.193:40479)
0812 07:05:31.759137 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:31.759143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:31.759162 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:05:31.759218 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:31.759224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:31.759302 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:31.759428 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:31.761199 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.762531 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.764375 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.765172 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.765306 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:31.768600 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:31.768615 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:31.768637 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:31.768688 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:31.773359 (+ 4671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:31.774204 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:31.774212 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:31.774216 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:31.774407 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:31.774805 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:31.774810 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:31.774813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:31.775256 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:31.775627 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:31.775907 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 07:05:31.776186 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:05:31.777638 4124 raft_consensus.cc:2229] T 35cbf50e0a76451289797386041ca0eb P 45e457a2ac3947068168483529c63743 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:05:31.777709 4125 tablet_replica.cc:330] T 35cbf50e0a76451289797386041ca0eb P 65a743ddefbc40fbb298aec04d085a7e: stopping tablet replica
I20260812 07:05:31.778012 4124 raft_consensus.cc:2258] T 35cbf50e0a76451289797386041ca0eb P 45e457a2ac3947068168483529c63743 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:05:31.778030 4125 raft_consensus.cc:2229] T 35cbf50e0a76451289797386041ca0eb P 65a743ddefbc40fbb298aec04d085a7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:05:31.778368 3463 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:05:31.778956 4125 raft_consensus.cc:2258] T 35cbf50e0a76451289797386041ca0eb P 65a743ddefbc40fbb298aec04d085a7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:05:31.779893 4124 ts_tablet_manager.cc:1905] T 35cbf50e0a76451289797386041ca0eb P 45e457a2ac3947068168483529c63743: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:05:31.781213 4125 ts_tablet_manager.cc:1905] T 35cbf50e0a76451289797386041ca0eb P 65a743ddefbc40fbb298aec04d085a7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:05:31.786538 4124 ts_tablet_manager.cc:1918] T 35cbf50e0a76451289797386041ca0eb P 45e457a2ac3947068168483529c63743: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:05:31.786700 4124 log.cc:1196] T 35cbf50e0a76451289797386041ca0eb P 45e457a2ac3947068168483529c63743: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/35cbf50e0a76451289797386041ca0eb
I20260812 07:05:31.787206 4124 ts_tablet_manager.cc:1939] T 35cbf50e0a76451289797386041ca0eb P 45e457a2ac3947068168483529c63743: Deleting consensus metadata
I20260812 07:05:31.788023 4125 ts_tablet_manager.cc:1918] T 35cbf50e0a76451289797386041ca0eb P 65a743ddefbc40fbb298aec04d085a7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:05:31.788231 4125 log.cc:1196] T 35cbf50e0a76451289797386041ca0eb P 65a743ddefbc40fbb298aec04d085a7e: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/35cbf50e0a76451289797386041ca0eb
I20260812 07:05:31.788779 4125 ts_tablet_manager.cc:1939] T 35cbf50e0a76451289797386041ca0eb P 65a743ddefbc40fbb298aec04d085a7e: Deleting consensus metadata
I20260812 07:05:31.788968 3040 catalog_manager.cc:4841] TS 45e457a2ac3947068168483529c63743 (127.11.192.194:36327): tablet 35cbf50e0a76451289797386041ca0eb (table test-table [id=749f4530633443278662cb62c3e600ab]) successfully deleted
I20260812 07:05:31.790123 3040 catalog_manager.cc:4841] TS 65a743ddefbc40fbb298aec04d085a7e (127.11.192.195:40087): tablet 35cbf50e0a76451289797386041ca0eb (table test-table [id=749f4530633443278662cb62c3e600ab]) successfully deleted
I20260812 07:05:31.879180 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 3163
W20260812 07:05:31.905143 3041 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.192.193:40479 (error 108)
I20260812 07:05:31.905356 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 3476
I20260812 07:05:31.907091 4077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:31.906668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:40479 (local address 127.0.0.1:50262)
0812 07:05:31.906836 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:31.906944 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:40479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 07:05:31.907433 3041 catalog_manager.cc:4584] TS 5935e032929a495da3d36d526b0ab0ed (127.11.192.193:40479): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 35cbf50e0a76451289797386041ca0eb: Network error: Client connection negotiation failed: client connection to 127.11.192.193:40479: connect: Connection refused (error 111)
I20260812 07:05:31.918366 4078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:31.906066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45300 (local address 127.11.192.194:36327)
0812 07:05:31.906236 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:31.906241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:31.906333 (+ 92us) server_negotiation.cc:407] Connection header received
0812 07:05:31.906495 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:31.906499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:31.906563 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:31.906671 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:31.908168 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.908871 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.910163 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:31.910567 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:31.910678 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:31.913058 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:31.913072 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:31.913086 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:31.913123 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:31.915645 (+ 2522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:31.916339 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:31.916347 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:31.916349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:31.916496 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:31.916846 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:31.916850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:31.916852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:31.917215 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:31.917506 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:31.917817 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 07:05:31.918055 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:05:31.918957 3750 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:05:31.960817 4077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:31.960430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:40479 (local address 127.0.0.1:50274)
0812 07:05:31.960600 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:31.960691 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:40479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 07:05:31.961798 4077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:31.961463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:40479 (local address 127.0.0.1:50276)
0812 07:05:31.961613 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:31.961694 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:40479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 07:05:32.009420 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 3476
I20260812 07:05:32.022228 4077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:32.021649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:40479 (local address 127.0.0.1:50278)
0812 07:05:32.021907 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:32.022027 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:40479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260812 07:05:32.023240 4077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:32.022870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:40479 (local address 127.0.0.1:50284)
0812 07:05:32.023018 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:32.023100 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:40479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 07:05:32.034700 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 3768
I20260812 07:05:32.047511 4085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:32.035419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59406 (local address 127.11.192.195:40087)
0812 07:05:32.035592 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:32.035596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:32.035614 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:05:32.035724 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:32.035728 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:32.035790 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:32.035900 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:32.037222 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:32.037935 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:32.039207 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:32.039570 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:32.039655 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:32.042290 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:32.042304 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:32.042321 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:32.042359 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:32.044936 (+ 2577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:32.045567 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:32.045575 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:32.045577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:32.045709 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:32.046069 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:32.046073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:32.046075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:32.046421 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:32.046709 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:32.047014 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 07:05:32.047214 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:05:32.048122 4014 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:05:32.103609 4077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:32.103182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:40479 (local address 127.0.0.1:50296)
0812 07:05:32.103377 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:32.103461 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:40479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 07:05:32.104707 4077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:32.104342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:40479 (local address 127.0.0.1:50300)
0812 07:05:32.104535 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:32.104604 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:40479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260812 07:05:32.134400 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 3768
I20260812 07:05:32.159933 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 2998
I20260812 07:05:32.161078 3148 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:05:32.269855 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 2998
I20260812 07:05:32.307106 12035 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 07:05:32.650703 12035 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 07:05:32.650925 12035 mini_ranger.cc:66] Stopping Ranger...
I20260812 07:05:32.994393 12035 mini_ranger.cc:68] Stopped Ranger
2026-08-12 07:05:32.994 UTC [1125] LOG: received smart shutdown request
2026-08-12 07:05:32.997 UTC [1125] LOG: background worker "logical replication launcher" (PID 1144) exited with exit code 1
2026-08-12 07:05:32.997 UTC [1132] LOG: shutting down
2026-08-12 07:05:33.128 UTC [1125] LOG: database system is shut down
2026-08-12T07:05:33Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49753 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:05:33 dist-test-slave-lgr5 krb5kdc[4144](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:05:33 dist-test-slave-lgr5 krb5kdc[4144](info): set up 2 sockets
Aug 12 07:05:33 dist-test-slave-lgr5 krb5kdc[4144](info): commencing operation
krb5kdc: starting...
W20260812 07:05:34.492666 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:05:34 dist-test-slave-lgr5 krb5kdc[4144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:05:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:05:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:05:34.740195 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:38651
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:37003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:38651,127.11.192.253:41723,127.11.192.252:34747
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:05:35.057302 4172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:35.057559 4172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:35.057657 4172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:35.064399 4172 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:05:35.064507 4172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:35.064584 4172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:35.064659 4172 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:05:35.064718 4172 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:05:35.073843 4172 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:37003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:38651,127.11.192.253:41723,127.11.192.252:34747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:38651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:35.074957 4172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:35.076202 4172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:05:35.086971 4181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:35.087005 4185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:35.087159 4182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:35.087750 4172 server_base.cc:1034] running on GCE node
I20260812 07:05:35.088534 4172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:35.089713 4172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:35.090907 4172 hybrid_clock.cc:648] HybridClock initialized: now 1786518335090850 us; error 44 us; skew 500 ppm
I20260812 07:05:35.091207 4172 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:35 dist-test-slave-lgr5 krb5kdc[4144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518335, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:35.094064 4172 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:05:35.095340 4172 webserver.cc:460] Webserver started at http://127.11.192.254:46719/ using document root <none> and password file <none>
I20260812 07:05:35.095955 4172 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:35.096071 4172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:05:35.096415 4172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:05:35.099145 4172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "eeb817af5aba45e28e8f02b725b17a78"
format_stamp: "Formatted at 2026-08-12 07:05:35 on dist-test-slave-lgr5"
server_key: "98b5ef7ab327b2f999019462c648f988"
server_key_iv: "2590afe8658fed0d0bd03f64d76f149c"
server_key_version: "encryptionkey@0"
I20260812 07:05:35.099897 4172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "eeb817af5aba45e28e8f02b725b17a78"
format_stamp: "Formatted at 2026-08-12 07:05:35 on dist-test-slave-lgr5"
server_key: "98b5ef7ab327b2f999019462c648f988"
server_key_iv: "2590afe8658fed0d0bd03f64d76f149c"
server_key_version: "encryptionkey@0"
I20260812 07:05:35.104643 4172 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 07:05:35.107767 4196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:35.109091 4172 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:05:35.109231 4172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "eeb817af5aba45e28e8f02b725b17a78"
format_stamp: "Formatted at 2026-08-12 07:05:35 on dist-test-slave-lgr5"
server_key: "98b5ef7ab327b2f999019462c648f988"
server_key_iv: "2590afe8658fed0d0bd03f64d76f149c"
server_key_version: "encryptionkey@0"
I20260812 07:05:35.109392 4172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:35.125556 4172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:35.129357 4172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:35.129604 4172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:35.146555 4172 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:38651
I20260812 07:05:35.146598 4289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:38651 every 8 connection(s)
I20260812 07:05:35.147796 4172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:05:35.149209 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 4172
I20260812 07:05:35.149439 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:05:35.150271 12035 external_mini_cluster.cc:1428] Setting key b29fc550990d98d3b32bbe48ec62d3a2
I20260812 07:05:35.152052 4290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:05:35.160334 4290 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:05:35 dist-test-slave-lgr5 krb5kdc[4144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518335, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.192.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.253@KRBTEST.COM" created.
I20260812 07:05:35.182518 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.163719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:38651 (local address 127.0.0.1:35384)
0812 07:05:35.164929 (+ 1210us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.164970 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:35.166766 (+ 1796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:35.167572 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:35.167579 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:35.168283 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:05:35.168638 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:35.168656 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:35.170245 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:35.170249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:35.171378 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:35.171392 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:35.172348 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:35.172357 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:35.172531 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:35.173193 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:05:35.173217 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:05:35.175535 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:05:35.178762 (+ 3227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:35.178772 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:35.178779 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:35.179216 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:35.179717 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:35.179720 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:35.179722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:35.179875 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:35.180807 (+ 932us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:05:35.180813 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:05:35.181015 (+ 202us) client_negotiation.cc:769] Sending connection context
0812 07:05:35.181208 (+ 193us) client_negotiation.cc:240] Negotiation successful
0812 07:05:35.181550 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"spinlock_wait_cycles":18688,"thread_start_us":799,"threads_started":1}
I20260812 07:05:35.182560 4296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.163718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35384 (local address 127.11.192.254:38651)
0812 07:05:35.164615 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:35.164630 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:35.165087 (+ 457us) server_negotiation.cc:407] Connection header received
0812 07:05:35.167117 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:35.167123 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:35.167344 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:35.167528 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:35.168844 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:35.170079 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:35.171522 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:35.172110 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:35.172312 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:35.175763 (+ 3451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:35.175772 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:35.175782 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:35.175815 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:35.178622 (+ 2807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:35.179376 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:35.179385 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:35.179390 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:35.179601 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:35.179985 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:35.179991 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:35.179995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:35.180674 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:35.180884 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:35.181394 (+ 510us) server_negotiation.cc:299] Negotiation successful
0812 07:05:35.181709 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":668,"thread_start_us":414,"threads_started":1}
I20260812 07:05:35.185365 4290 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:05:35.186373 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.185947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41723 (local address 127.0.0.1:34922)
0812 07:05:35.186107 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.186233 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 07:05:35.186623 4200 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.192.253:41723: connect: Connection refused (error 111)
I20260812 07:05:35.188126 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.187738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41723 (local address 127.0.0.1:34938)
0812 07:05:35.187903 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.187977 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 07:05:35.188302 4290 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.253:41723: Network error: Client connection negotiation failed: client connection to 127.11.192.253:41723: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.253.keytab.
Entry for principal kudu/127.11.192.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.253.keytab.
I20260812 07:05:35.206890 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.253:41723
--webserver_interface=127.11.192.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.253.keytab
--principal=kudu/127.11.192.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:37003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:38651,127.11.192.253:41723,127.11.192.252:34747
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:05:35.237613 4290 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } attempt: 1
I20260812 07:05:35.238780 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.238249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41723 (local address 127.0.0.1:34950)
0812 07:05:35.238480 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.238622 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 07:05:35.239782 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.239396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41723 (local address 127.0.0.1:34960)
0812 07:05:35.239545 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.239629 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 07:05:35.239981 4290 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.253:41723: Network error: Client connection negotiation failed: client connection to 127.11.192.253:41723: connect: Connection refused (error 111)
I20260812 07:05:35.308265 4290 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } attempt: 2
I20260812 07:05:35.309669 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.309060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41723 (local address 127.0.0.1:34962)
0812 07:05:35.309346 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.309469 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 07:05:35.310772 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.310314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41723 (local address 127.0.0.1:34966)
0812 07:05:35.310487 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.310602 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 07:05:35.311017 4290 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.253:41723: Network error: Client connection negotiation failed: client connection to 127.11.192.253:41723: connect: Connection refused (error 111)
I20260812 07:05:35.402297 4290 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } attempt: 3
I20260812 07:05:35.403491 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.402975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41723 (local address 127.0.0.1:34974)
0812 07:05:35.403199 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.403317 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 07:05:35.404596 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.404174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41723 (local address 127.0.0.1:34988)
0812 07:05:35.404329 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.404436 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 07:05:35.404789 4290 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.253:41723: Network error: Client connection negotiation failed: client connection to 127.11.192.253:41723: connect: Connection refused (error 111)
W20260812 07:05:35.527079 4307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:35.527335 4307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:35.527463 4307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:35.534376 4307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:05:35.534473 4307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:35.534549 4307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:35.534624 4307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:05:35.534683 4307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:05:35.543977 4307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:37003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:38651,127.11.192.253:41723,127.11.192.252:34747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.253.keytab
--principal=kudu/127.11.192.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.253:41723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.192.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:35.545142 4307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:35.546345 4307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:05:35.548043 4290 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } attempt: 4
I20260812 07:05:35.549291 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.548697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41723 (local address 127.0.0.1:34992)
0812 07:05:35.549007 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.549119 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260812 07:05:35.550294 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.549879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41723 (local address 127.0.0.1:34994)
0812 07:05:35.550027 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.550136 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 07:05:35.550498 4290 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.253:41723: Network error: Client connection negotiation failed: client connection to 127.11.192.253:41723: connect: Connection refused (error 111)
W20260812 07:05:35.557884 4314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:35.557817 4321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:35.557879 4317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:35.558683 4307 server_base.cc:1034] running on GCE node
I20260812 07:05:35.559397 4307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:35.560671 4307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:35.561863 4307 hybrid_clock.cc:648] HybridClock initialized: now 1786518335561820 us; error 38 us; skew 500 ppm
I20260812 07:05:35.562175 4307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:35 dist-test-slave-lgr5 krb5kdc[4144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518335, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:35.568197 4307 init.cc:377] Logged in from keytab as kudu/127.11.192.253@KRBTEST.COM (short username kudu)
I20260812 07:05:35.569521 4307 webserver.cc:460] Webserver started at http://127.11.192.253:40487/ using document root <none> and password file <none>
I20260812 07:05:35.570145 4307 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:35.570259 4307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:05:35.570541 4307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:05:35.573349 4307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-1/data/instance:
uuid: "ca5c9e038aee4b2397d27e372ac57214"
format_stamp: "Formatted at 2026-08-12 07:05:35 on dist-test-slave-lgr5"
server_key: "91e89f91a4707f11c44fa1683ba70804"
server_key_iv: "f75aec6f52cd68cb70acff129894ec68"
server_key_version: "encryptionkey@0"
I20260812 07:05:35.574127 4307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-1/wal/instance:
uuid: "ca5c9e038aee4b2397d27e372ac57214"
format_stamp: "Formatted at 2026-08-12 07:05:35 on dist-test-slave-lgr5"
server_key: "91e89f91a4707f11c44fa1683ba70804"
server_key_iv: "f75aec6f52cd68cb70acff129894ec68"
server_key_version: "encryptionkey@0"
I20260812 07:05:35.578896 4307 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 07:05:35.582034 4330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:35.583218 4307 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:05:35.583376 4307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-1/wal
uuid: "ca5c9e038aee4b2397d27e372ac57214"
format_stamp: "Formatted at 2026-08-12 07:05:35 on dist-test-slave-lgr5"
server_key: "91e89f91a4707f11c44fa1683ba70804"
server_key_iv: "f75aec6f52cd68cb70acff129894ec68"
server_key_version: "encryptionkey@0"
I20260812 07:05:35.583552 4307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:35.598683 4307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:35.602226 4307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:35.602469 4307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:35.619468 4307 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.253:41723
I20260812 07:05:35.619503 4430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.253:41723 every 8 connection(s)
I20260812 07:05:35.620748 4307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-1/data/info.pb
I20260812 07:05:35.624177 4431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:05:35.627846 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 4307
I20260812 07:05:35.628053 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-1/wal/instance
I20260812 07:05:35.628646 12035 external_mini_cluster.cc:1428] Setting key bbc2b5bb8e5a553bee658b42118d222e
I20260812 07:05:35.631673 4431 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:05:35 dist-test-slave-lgr5 krb5kdc[4144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518335, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.253@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:05:35.650833 4296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.634663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35398 (local address 127.11.192.254:38651)
0812 07:05:35.634870 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:35.634877 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:35.636016 (+ 1139us) server_negotiation.cc:407] Connection header received
0812 07:05:35.637855 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:35.637860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:35.637939 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:35.638044 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:35.639421 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:35.640230 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:35.641725 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:35.642131 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:35.642240 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:35.645473 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:35.645478 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:35.645481 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:35.645519 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:35.647915 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:35.648640 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:35.648644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:35.648646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:35.648761 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:35.649161 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:35.649165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:35.649167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:35.649470 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:35.649628 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:35.650295 (+ 667us) server_negotiation.cc:299] Negotiation successful
0812 07:05:35.650530 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:05:35.651662 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.635070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:38651 (local address 127.0.0.1:35398)
0812 07:05:35.635836 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.635882 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:35.637603 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:35.638225 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:35.638232 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:35.638709 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:05:35.639244 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:35.639261 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:35.640394 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:35.640398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:35.641583 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:35.641597 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:35.642340 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:35.642348 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:35.642550 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:35.643147 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:05:35.643169 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:05:35.645253 (+ 2084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:05:35.648054 (+ 2801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:35.648063 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:35.648106 (+ 43us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:35.648529 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:35.648863 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:35.648868 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:35.648872 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:35.649047 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:35.649618 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:05:35.649626 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:05:35.649916 (+ 290us) client_negotiation.cc:769] Sending connection context
0812 07:05:35.650156 (+ 240us) client_negotiation.cc:240] Negotiation successful
0812 07:05:35.650603 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":326,"threads_started":1}
WARNING: no policy specified for kudu/127.11.192.252@KRBTEST.COM; defaulting to no policy
I20260812 07:05:35.653425 4431 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.252@KRBTEST.COM" created.
Aug 12 07:05:35 dist-test-slave-lgr5 krb5kdc[4144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518335, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.253@KRBTEST.COM for kudu/127.11.192.253@KRBTEST.COM
I20260812 07:05:35.667469 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.653983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41723 (local address 127.0.0.1:35006)
0812 07:05:35.654129 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.654143 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:35.654220 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:35.655175 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:35.655178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:35.655510 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:05:35.655639 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:35.655648 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:35.657330 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:35.657334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:35.658127 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:35.658139 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:35.659115 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:35.659119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:35.659283 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:35.659752 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:05:35.659769 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:05:35.661586 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:05:35.665043 (+ 3457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:35.665049 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:35.665052 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:35.665528 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:35.665855 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:35.665860 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:35.665863 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:35.666049 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:35.666585 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:05:35.666590 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:05:35.666725 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 07:05:35.666875 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 07:05:35.667130 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 07:05:35.667505 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.654220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35006 (local address 127.11.192.253:41723)
0812 07:05:35.654795 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:35.654801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:35.654823 (+ 22us) server_negotiation.cc:407] Connection header received
0812 07:05:35.654861 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:35.654867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:35.655027 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:35.655190 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:35.655784 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:35.657194 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:35.658263 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:35.658972 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:35.659099 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:35.661788 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:35.661801 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:35.661815 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:35.661847 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:35.664918 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:35.665640 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:35.665646 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:35.665648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:35.665751 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:35.666152 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:35.666155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:35.666157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:35.666460 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:35.666605 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:35.666995 (+ 390us) server_negotiation.cc:299] Negotiation successful
0812 07:05:35.667232 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":445,"thread_start_us":322,"threads_started":1}
I20260812 07:05:35.669824 4431 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:05:35.670837 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.670363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:34747 (local address 127.0.0.1:40550)
0812 07:05:35.670552 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.670666 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 07:05:35.671065 4332 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
I20260812 07:05:35.672607 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.672252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:34747 (local address 127.0.0.1:40566)
0812 07:05:35.672400 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.672499 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 07:05:35.672803 4431 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:34747: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.252.keytab.
Entry for principal kudu/127.11.192.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.252.keytab.
I20260812 07:05:35.684464 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.252:34747
--webserver_interface=127.11.192.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.252.keytab
--principal=kudu/127.11.192.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:37003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:38651,127.11.192.253:41723,127.11.192.252:34747
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:05:35.722118 4431 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } attempt: 1
I20260812 07:05:35.723348 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.722831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:34747 (local address 127.0.0.1:40570)
0812 07:05:35.723040 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.723164 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 07:05:35.724335 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.723929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:34747 (local address 127.0.0.1:40584)
0812 07:05:35.724117 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.724203 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 07:05:35.724536 4431 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:34747: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
I20260812 07:05:35.792766 4431 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } attempt: 2
I20260812 07:05:35.793967 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.793426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:34747 (local address 127.0.0.1:40598)
0812 07:05:35.793645 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.793759 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 07:05:35.795014 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.794605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:34747 (local address 127.0.0.1:40610)
0812 07:05:35.794773 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.794864 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 07:05:35.795300 4431 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:34747: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
I20260812 07:05:35.849779 4290 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } attempt: 5
Aug 12 07:05:35 dist-test-slave-lgr5 krb5kdc[4144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518335, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for kudu/127.11.192.253@KRBTEST.COM
I20260812 07:05:35.862545 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.850554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41723 (local address 127.0.0.1:35012)
0812 07:05:35.850749 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.850769 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:35.850898 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:35.851298 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:35.851304 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:35.851696 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:05:35.851885 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:35.851895 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:35.853002 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:35.853007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:35.853882 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:35.853896 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:35.854664 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:35.854671 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:35.854840 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:35.855443 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:05:35.855460 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:05:35.857437 (+ 1977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:05:35.860214 (+ 2777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:35.860219 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:35.860224 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:35.860579 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:35.860929 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:35.860934 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:35.860938 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:35.861079 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:35.861581 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:05:35.861586 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:05:35.861735 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 07:05:35.861876 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 07:05:35.862146 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 07:05:35.862537 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.850753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35012 (local address 127.11.192.253:41723)
0812 07:05:35.850953 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:35.850959 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:35.850979 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:05:35.851058 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:35.851062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:35.851139 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:35.851226 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:35.852063 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:35.852859 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:35.854036 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:35.854504 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:35.854614 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:35.857626 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:35.857632 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:35.857635 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:35.857669 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:35.860058 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:35.860687 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:35.860693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:35.860697 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:35.860823 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:35.861180 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:35.861186 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:35.861190 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:35.861465 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:35.861600 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:35.862007 (+ 407us) server_negotiation.cc:299] Negotiation successful
0812 07:05:35.862229 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:05:35.863754 4290 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:05:35.864852 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.864484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:34747 (local address 127.0.0.1:40620)
0812 07:05:35.864647 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.864723 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 07:05:35.865773 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.865460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:34747 (local address 127.0.0.1:40626)
0812 07:05:35.865597 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.865681 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 07:05:35.865976 4290 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:34747: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
I20260812 07:05:35.886516 4431 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } attempt: 3
I20260812 07:05:35.887624 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.887073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:34747 (local address 127.0.0.1:40632)
0812 07:05:35.887290 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.887412 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 07:05:35.888716 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.888294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:34747 (local address 127.0.0.1:40636)
0812 07:05:35.888527 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.888604 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260812 07:05:35.888934 4431 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:34747: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
I20260812 07:05:35.917176 4290 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } attempt: 1
I20260812 07:05:35.918197 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.917781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:34747 (local address 127.0.0.1:40640)
0812 07:05:35.917947 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.918046 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 07:05:35.919088 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.918761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:34747 (local address 127.0.0.1:40652)
0812 07:05:35.918902 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.918978 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 07:05:35.919274 4290 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:34747: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
I20260812 07:05:35.987470 4290 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } attempt: 2
I20260812 07:05:35.988565 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.988097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:34747 (local address 127.0.0.1:40668)
0812 07:05:35.988323 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.988411 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 07:05:35.989503 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:35.989188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:34747 (local address 127.0.0.1:40674)
0812 07:05:35.989324 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:35.989407 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 07:05:35.989691 4290 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:34747: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
W20260812 07:05:36.008419 4441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:36.008690 4441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:36.008829 4441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:36.015981 4441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:05:36.016103 4441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:36.016183 4441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:36.016263 4441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:05:36.016323 4441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:05:36.025952 4441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:37003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:38651,127.11.192.253:41723,127.11.192.252:34747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.252.keytab
--principal=kudu/127.11.192.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.252:34747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.192.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:36.027172 4441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:36.028501 4441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:05:36.032136 4431 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } attempt: 4
I20260812 07:05:36.033298 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:36.032796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:34747 (local address 127.0.0.1:40680)
0812 07:05:36.033013 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:36.033099 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 07:05:36.034510 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:36.034063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:34747 (local address 127.0.0.1:40686)
0812 07:05:36.034273 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:36.034357 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260812 07:05:36.034827 4431 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:34747: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
W20260812 07:05:36.040056 4457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:36.040050 4460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:36.040117 4456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:36.040841 4441 server_base.cc:1034] running on GCE node
I20260812 07:05:36.041545 4441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:36.042779 4441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:36.043972 4441 hybrid_clock.cc:648] HybridClock initialized: now 1786518336043921 us; error 45 us; skew 500 ppm
I20260812 07:05:36.044312 4441 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:36 dist-test-slave-lgr5 krb5kdc[4144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518336, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:36.047186 4441 init.cc:377] Logged in from keytab as kudu/127.11.192.252@KRBTEST.COM (short username kudu)
I20260812 07:05:36.048532 4441 webserver.cc:460] Webserver started at http://127.11.192.252:34887/ using document root <none> and password file <none>
I20260812 07:05:36.049160 4441 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:36.049271 4441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:05:36.049551 4441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:05:36.052402 4441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-2/data/instance:
uuid: "85d78ecf62004006823ed80b0e7177ce"
format_stamp: "Formatted at 2026-08-12 07:05:36 on dist-test-slave-lgr5"
server_key: "62b42095c35c9c22052e6bc314c6e420"
server_key_iv: "e79cd91320d39d3f5ff8c13e0a821f74"
server_key_version: "encryptionkey@0"
I20260812 07:05:36.053187 4441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-2/wal/instance:
uuid: "85d78ecf62004006823ed80b0e7177ce"
format_stamp: "Formatted at 2026-08-12 07:05:36 on dist-test-slave-lgr5"
server_key: "62b42095c35c9c22052e6bc314c6e420"
server_key_iv: "e79cd91320d39d3f5ff8c13e0a821f74"
server_key_version: "encryptionkey@0"
I20260812 07:05:36.058171 4441 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:05:36.061295 4481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:36.062480 4441 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:05:36.062636 4441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-2/wal
uuid: "85d78ecf62004006823ed80b0e7177ce"
format_stamp: "Formatted at 2026-08-12 07:05:36 on dist-test-slave-lgr5"
server_key: "62b42095c35c9c22052e6bc314c6e420"
server_key_iv: "e79cd91320d39d3f5ff8c13e0a821f74"
server_key_version: "encryptionkey@0"
I20260812 07:05:36.062799 4441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:36.093793 4441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:36.095898 4290 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } attempt: 3
I20260812 07:05:36.097010 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:36.096548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:34747 (local address 127.0.0.1:40692)
0812 07:05:36.096735 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:36.096835 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 07:05:36.097779 4441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:36.098014 4441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:36.098096 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:36.097691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:34747 (local address 127.0.0.1:40696)
0812 07:05:36.097895 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:36.097983 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260812 07:05:36.098307 4290 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:34747: Network error: Client connection negotiation failed: client connection to 127.11.192.252:34747: connect: Connection refused (error 111)
I20260812 07:05:36.114655 4441 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.252:34747
I20260812 07:05:36.114693 4581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.252:34747 every 8 connection(s)
I20260812 07:05:36.115918 4441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-2/data/info.pb
I20260812 07:05:36.119446 4585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:05:36.126049 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 4441
I20260812 07:05:36.126219 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518136653111-12035-0/minicluster-data/master-2/wal/instance
I20260812 07:05:36.126641 12035 external_mini_cluster.cc:1428] Setting key 489e0abfe976b6082f0441e93eecce0a
I20260812 07:05:36.126396 4585 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:05:36 dist-test-slave-lgr5 krb5kdc[4144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518336, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.252@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
Aug 12 07:05:36 dist-test-slave-lgr5 krb5kdc[4144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.252@KRBTEST.COM
I20260812 07:05:36.145682 4296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:36.129125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35408 (local address 127.11.192.254:38651)
0812 07:05:36.129324 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:36.129331 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:36.130225 (+ 894us) server_negotiation.cc:407] Connection header received
0812 07:05:36.131193 (+ 968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:36.131198 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:36.131272 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:36.131357 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:36.133096 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.134106 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.136257 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.136684 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.136812 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:36.140163 (+ 3351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:36.140168 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:36.140172 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:36.140202 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:36.142809 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:36.143503 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:36.143508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:36.143510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:36.143634 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:36.144046 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:36.144050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:36.144052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:36.144339 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:36.144482 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:36.145115 (+ 633us) server_negotiation.cc:299] Negotiation successful
0812 07:05:36.145341 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 07:05:36.146631 4596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:36.129186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:38651 (local address 127.0.0.1:35408)
0812 07:05:36.130054 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:36.130107 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:36.130946 (+ 839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:36.131467 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:36.131475 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:36.132354 (+ 879us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:05:36.132919 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:36.132947 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.134277 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.134281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:36.136126 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:36.136142 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.136895 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.136900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:36.137072 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:36.137677 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:05:36.137699 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:05:36.139966 (+ 2267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:05:36.142925 (+ 2959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:36.142937 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:36.142947 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:36.143383 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:36.143741 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:36.143746 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:36.143753 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:36.143942 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:36.144467 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:05:36.144474 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:05:36.144752 (+ 278us) client_negotiation.cc:769] Sending connection context
0812 07:05:36.145001 (+ 249us) client_negotiation.cc:240] Negotiation successful
0812 07:05:36.145411 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"mutex_wait_us":25,"thread_start_us":412,"threads_started":1}
I20260812 07:05:36.147294 4595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:36.128387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40712 (local address 127.11.192.252:34747)
0812 07:05:36.129109 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:36.129122 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:36.129165 (+ 43us) server_negotiation.cc:407] Connection header received
0812 07:05:36.130933 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:36.130954 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:36.131199 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:36.131400 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:36.133342 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.134829 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.136729 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.137322 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.137429 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:36.141391 (+ 3962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:36.141399 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:36.141409 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:36.141437 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:36.143904 (+ 2467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:36.144597 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:36.144603 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:36.144605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:36.144730 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:36.145105 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:36.145109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:36.145111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:36.145437 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:36.145696 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:36.146661 (+ 965us) server_negotiation.cc:299] Negotiation successful
0812 07:05:36.146889 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":523,"thread_start_us":324,"threads_started":1}
I20260812 07:05:36.148159 4585 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:05:36 dist-test-slave-lgr5 krb5kdc[4144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518336, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.252@KRBTEST.COM for kudu/127.11.192.253@KRBTEST.COM
Aug 12 07:05:36 dist-test-slave-lgr5 krb5kdc[4144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:05:36.162303 4596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:36.148731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:41723 (local address 127.0.0.1:35018)
0812 07:05:36.149246 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:36.149263 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:36.149359 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:36.149729 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:36.149734 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:36.150129 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:05:36.150296 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:36.150307 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.151358 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.151361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:36.152379 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:36.152394 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.153679 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.153683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:36.153842 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:36.154294 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:05:36.154310 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:05:36.156446 (+ 2136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:05:36.159956 (+ 3510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:36.159960 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:36.159962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:36.160495 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:36.160859 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:36.160864 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:36.160868 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:36.161028 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:36.161594 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:05:36.161597 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:05:36.161741 (+ 144us) client_negotiation.cc:769] Sending connection context
0812 07:05:36.161906 (+ 165us) client_negotiation.cc:240] Negotiation successful
0812 07:05:36.162091 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413}
I20260812 07:05:36.162611 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:36.149070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35018 (local address 127.11.192.253:41723)
0812 07:05:36.149269 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:36.149275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:36.149296 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:05:36.149520 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:36.149527 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:36.149599 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:36.149681 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:36.150467 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.151202 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.152538 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.153176 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.153262 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:36.156632 (+ 3370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:36.156639 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:36.156644 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:36.156684 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:36.159832 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:36.160606 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:36.160611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:36.160613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:36.160742 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:36.161152 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:36.161157 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:36.161160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:36.161468 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:36.161630 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:36.162055 (+ 425us) server_negotiation.cc:299] Negotiation successful
0812 07:05:36.162258 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:05:36.163483 4585 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:05:36.165978 4296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:36.150013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35410 (local address 127.11.192.254:38651)
0812 07:05:36.150164 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:36.150170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:36.150237 (+ 67us) server_negotiation.cc:407] Connection header received
0812 07:05:36.150404 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:36.150409 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:36.150475 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:36.150575 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:36.151976 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.153147 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.154918 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.155540 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.155666 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:36.160168 (+ 4502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:36.160175 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:36.160179 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:36.160213 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:36.163278 (+ 3065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:36.163934 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:36.163940 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:36.163943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:36.164135 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:36.164552 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:36.164556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:36.164558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:36.164918 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:36.165120 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:36.165446 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 07:05:36.165677 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 07:05:36 dist-test-slave-lgr5 krb5kdc[4144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518336, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.252@KRBTEST.COM for kudu/127.11.192.252@KRBTEST.COM
Aug 12 07:05:36 dist-test-slave-lgr5 krb5kdc[4144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.253@KRBTEST.COM
I20260812 07:05:36.176144 4596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:36.164253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:34747 (local address 127.0.0.1:40728)
0812 07:05:36.164413 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:36.164431 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:36.164523 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:36.164797 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:36.164800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:36.165174 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:05:36.165369 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:36.165382 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.166484 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.166500 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:36.167646 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:36.167660 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.168344 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.168350 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:36.168544 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:36.169020 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:05:36.169034 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:05:36.171018 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:05:36.173901 (+ 2883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:36.173906 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:36.173910 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:36.174375 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:36.174703 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:36.174707 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:36.174711 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:36.174863 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:36.175398 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:05:36.175402 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:05:36.175523 (+ 121us) client_negotiation.cc:769] Sending connection context
0812 07:05:36.175626 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 07:05:36.175877 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 07:05:36.176407 4595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:36.164190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40728 (local address 127.11.192.252:34747)
0812 07:05:36.164458 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:36.164461 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:36.164475 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:05:36.164622 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:36.164627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:36.164692 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:36.164787 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:36.165512 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.166377 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.167767 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.168217 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.168324 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:36.171189 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:36.171194 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:36.171197 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:36.171224 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:36.173772 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:36.174485 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:36.174489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:36.174491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:36.174601 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:36.174976 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:36.174979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:36.174981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:36.175281 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:36.175418 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:36.175745 (+ 327us) server_negotiation.cc:299] Negotiation successful
0812 07:05:36.176024 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":191}
I20260812 07:05:36.181418 4585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce: Bootstrap starting.
I20260812 07:05:36.181578 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:36.167521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35028 (local address 127.11.192.253:41723)
0812 07:05:36.167662 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:36.167667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:36.167683 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:05:36.167817 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:36.167831 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:36.167889 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:36.167963 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:36.169251 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.169962 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.171552 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.172155 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.172278 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:36.176804 (+ 4526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:36.176809 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:36.176812 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:36.176840 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:36.179197 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:36.179913 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:36.179917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:36.179919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:36.180044 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:36.180467 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:36.180470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:36.180472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:36.180715 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:36.180912 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:36.181187 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 07:05:36.181336 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:05:36.185120 4585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce: Neither blocks nor log segments found. Creating new log.
I20260812 07:05:36.185912 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:05:36.186198 4585 log.cc:824] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce: Log is configured to *not* fsync() on all Append() calls
I20260812 07:05:36.189211 4585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce: No bootstrap required, opened a new log
I20260812 07:05:36.200253 4585 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb817af5aba45e28e8f02b725b17a78" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 } } peers { permanent_uuid: "ca5c9e038aee4b2397d27e372ac57214" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } } peers { permanent_uuid: "85d78ecf62004006823ed80b0e7177ce" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } }
I20260812 07:05:36.200652 4585 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:05:36.200773 4585 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85d78ecf62004006823ed80b0e7177ce, State: Initialized, Role: FOLLOWER
I20260812 07:05:36.202135 4585 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb817af5aba45e28e8f02b725b17a78" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 } } peers { permanent_uuid: "ca5c9e038aee4b2397d27e372ac57214" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } } peers { permanent_uuid: "85d78ecf62004006823ed80b0e7177ce" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } }
I20260812 07:05:36.209803 4296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:36.191254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35422 (local address 127.11.192.254:38651)
0812 07:05:36.191412 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:36.191418 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:36.191964 (+ 546us) server_negotiation.cc:407] Connection header received
0812 07:05:36.192234 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:36.192240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:36.192315 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:36.192385 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:36.194733 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.195863 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.198189 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.198904 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.198995 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:36.203163 (+ 4168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:36.203171 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:36.203176 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:36.203224 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:36.206580 (+ 3356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:36.207474 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:36.207479 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:36.207483 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:36.207644 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:36.208044 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:36.208049 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:36.208052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:36.208426 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:36.208656 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:36.209232 (+ 576us) server_negotiation.cc:299] Negotiation successful
0812 07:05:36.209459 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:05:36.213032 4617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb817af5aba45e28e8f02b725b17a78" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 } } peers { permanent_uuid: "ca5c9e038aee4b2397d27e372ac57214" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } } peers { permanent_uuid: "85d78ecf62004006823ed80b0e7177ce" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } } }
I20260812 07:05:36.214412 4585 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:05:36.214870 4617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:05:36.218531 4595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:36.191719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40744 (local address 127.11.192.252:34747)
0812 07:05:36.191884 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:36.191890 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:36.201592 (+ 9702us) server_negotiation.cc:407] Connection header received
0812 07:05:36.201742 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:36.201746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:36.201810 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:36.201894 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:36.203819 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.204980 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.207363 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.208146 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.208257 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:36.211605 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:36.211612 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:36.211617 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:36.211655 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:36.215032 (+ 3377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:36.216010 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:36.216016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:36.216020 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:36.216260 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:36.216702 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:36.216708 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:36.216711 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:36.217142 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:36.217380 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:36.217961 (+ 581us) server_negotiation.cc:299] Negotiation successful
0812 07:05:36.218170 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 07:05:36.220728 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:36.192462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35034 (local address 127.11.192.253:41723)
0812 07:05:36.192606 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:36.192611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:36.204475 (+ 11864us) server_negotiation.cc:407] Connection header received
0812 07:05:36.204824 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:36.204829 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:36.205145 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:36.205221 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:36.207112 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.208135 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.210334 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.210862 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.210932 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:36.214940 (+ 4008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:36.214948 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:36.214952 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:36.214989 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:36.218000 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:36.218797 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:36.218804 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:36.218807 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:36.218972 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:36.219346 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:36.219352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:36.219355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:36.219627 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:36.219854 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:36.220203 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 07:05:36.220418 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 07:05:36.234632 4644 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:05:36.234761 4644 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:05:36.231087 4441 master_runner.cc:186] Error getting master registration for 127.11.192.254:38651: OK, instance_id { permanent_uuid: "eeb817af5aba45e28e8f02b725b17a78" instance_seqno: 1786518335141778 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 07:05:36.239455 4441 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 07:05:36.275506 4290 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } attempt: 4
Aug 12 07:05:36 dist-test-slave-lgr5 krb5kdc[4144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518335, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for kudu/127.11.192.252@KRBTEST.COM
I20260812 07:05:36.288568 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:36.276026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:34747 (local address 127.0.0.1:40750)
0812 07:05:36.276213 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:36.276233 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:36.276340 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:36.276670 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:36.276674 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:36.277073 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:05:36.277233 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:36.277243 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.278294 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.278300 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:36.279279 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:36.279292 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.279986 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.280000 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:36.280201 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:36.280745 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:05:36.280762 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:05:36.282701 (+ 1939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:05:36.286154 (+ 3453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:36.286160 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:36.286164 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:36.286575 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:36.286924 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:36.286929 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:36.286932 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:36.287079 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:36.287577 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:05:36.287581 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:05:36.287742 (+ 161us) client_negotiation.cc:769] Sending connection context
0812 07:05:36.287877 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 07:05:36.288184 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 07:05:36.288579 4595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:36.276114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40750 (local address 127.11.192.252:34747)
0812 07:05:36.276272 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:36.276278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:36.276294 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:05:36.276449 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:36.276454 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:36.276547 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:36.276630 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:36.277374 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.278174 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.279416 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.279854 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.279962 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:36.282863 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:36.282868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:36.282872 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:36.282899 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:36.286030 (+ 3131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:36.286679 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:36.286685 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:36.286688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:36.286823 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:36.287183 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:36.287187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:36.287189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:36.287455 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:36.287622 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:36.287988 (+ 366us) server_negotiation.cc:299] Negotiation successful
0812 07:05:36.288231 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:05:36.294189 4290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78: Bootstrap starting.
I20260812 07:05:36.298211 4290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78: Neither blocks nor log segments found. Creating new log.
I20260812 07:05:36.299479 4290 log.cc:824] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78: Log is configured to *not* fsync() on all Append() calls
I20260812 07:05:36.302469 4290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78: No bootstrap required, opened a new log
I20260812 07:05:36.313054 4290 raft_consensus.cc:348] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb817af5aba45e28e8f02b725b17a78" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 } } peers { permanent_uuid: "ca5c9e038aee4b2397d27e372ac57214" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } } peers { permanent_uuid: "85d78ecf62004006823ed80b0e7177ce" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } }
I20260812 07:05:36.313418 4290 raft_consensus.cc:374] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:05:36.313534 4290 raft_consensus.cc:729] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eeb817af5aba45e28e8f02b725b17a78, State: Initialized, Role: FOLLOWER
I20260812 07:05:36.314158 4290 consensus_queue.cc:260] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb817af5aba45e28e8f02b725b17a78" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 } } peers { permanent_uuid: "ca5c9e038aee4b2397d27e372ac57214" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } } peers { permanent_uuid: "85d78ecf62004006823ed80b0e7177ce" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } }
I20260812 07:05:36.315014 4654 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb817af5aba45e28e8f02b725b17a78" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 } } peers { permanent_uuid: "ca5c9e038aee4b2397d27e372ac57214" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } } peers { permanent_uuid: "85d78ecf62004006823ed80b0e7177ce" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } } }
I20260812 07:05:36.315376 4654 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:05:36.316332 4290 sys_catalog.cc:564] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 07:05:36.326200 4671 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:05:36.326330 4671 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:05:36.334056 4431 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } attempt: 5
I20260812 07:05:36.325961 4172 master_runner.cc:186] Error getting master registration for 127.11.192.253:41723: OK, instance_id { permanent_uuid: "ca5c9e038aee4b2397d27e372ac57214" instance_seqno: 1786518335614830 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 07:05:36.338045 4172 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 12 07:05:36 dist-test-slave-lgr5 krb5kdc[4144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518335, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.253@KRBTEST.COM for kudu/127.11.192.252@KRBTEST.COM
W20260812 07:05:36.349965 4492 tablet.cc:2369] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:05:36.353559 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:36.334591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:34747 (local address 127.0.0.1:40754)
0812 07:05:36.334931 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:36.334951 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:36.335071 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:36.335962 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:36.335968 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:36.336535 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:05:36.336755 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:36.336769 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.338201 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.338207 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:36.339636 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:36.339652 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.342103 (+ 2451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.342109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:36.342328 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:36.343012 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:05:36.343037 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:05:36.345739 (+ 2702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:05:36.350803 (+ 5064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:36.350808 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:36.350812 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:36.351281 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:36.351711 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:36.351716 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:36.351720 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:36.351920 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:36.352584 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:05:36.352590 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:05:36.352752 (+ 162us) client_negotiation.cc:769] Sending connection context
0812 07:05:36.352926 (+ 174us) client_negotiation.cc:240] Negotiation successful
0812 07:05:36.353196 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204}
I20260812 07:05:36.353983 4595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:36.335276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40754 (local address 127.11.192.252:34747)
0812 07:05:36.335504 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:36.335510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:36.335529 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:05:36.335582 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:36.335587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:36.335659 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:36.335755 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:36.336936 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.338061 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.339792 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:36.340542 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:36.340670 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:36.345947 (+ 5277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:36.345954 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:36.345959 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:36.346014 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:36.349606 (+ 3592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:36.351410 (+ 1804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:36.351416 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:36.351420 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:36.351586 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:36.352045 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:36.352050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:36.352053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:36.352450 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:36.352630 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:36.353073 (+ 443us) server_negotiation.cc:299] Negotiation successful
0812 07:05:36.353314 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 07:05:36.359627 4431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214: Bootstrap starting.
I20260812 07:05:36.364245 4431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214: Neither blocks nor log segments found. Creating new log.
I20260812 07:05:36.365538 4431 log.cc:824] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214: Log is configured to *not* fsync() on all Append() calls
I20260812 07:05:36.368942 4431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214: No bootstrap required, opened a new log
I20260812 07:05:36.379698 4431 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb817af5aba45e28e8f02b725b17a78" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 } } peers { permanent_uuid: "ca5c9e038aee4b2397d27e372ac57214" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } } peers { permanent_uuid: "85d78ecf62004006823ed80b0e7177ce" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } }
I20260812 07:05:36.380095 4431 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:05:36.380224 4431 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5c9e038aee4b2397d27e372ac57214, State: Initialized, Role: FOLLOWER
I20260812 07:05:36.380934 4431 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb817af5aba45e28e8f02b725b17a78" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 } } peers { permanent_uuid: "ca5c9e038aee4b2397d27e372ac57214" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } } peers { permanent_uuid: "85d78ecf62004006823ed80b0e7177ce" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } }
W20260812 07:05:36.382009 4206 tablet.cc:2369] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:05:36.381913 4677 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb817af5aba45e28e8f02b725b17a78" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 } } peers { permanent_uuid: "ca5c9e038aee4b2397d27e372ac57214" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } } peers { permanent_uuid: "85d78ecf62004006823ed80b0e7177ce" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } } }
I20260812 07:05:36.382433 4677 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:05:36.383081 4431 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:05:36.403916 4307 master_runner.cc:263] No leader master found from master ca5c9e038aee4b2397d27e372ac57214
I20260812 07:05:36.404122 4307 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 07:05:36.406772 4694 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:05:36.406975 4694 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:05:36.409199 4677 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:05:36.409453 4677 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb817af5aba45e28e8f02b725b17a78" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 } } peers { permanent_uuid: "ca5c9e038aee4b2397d27e372ac57214" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } } peers { permanent_uuid: "85d78ecf62004006823ed80b0e7177ce" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } }
I20260812 07:05:36.410907 4677 leader_election.cc:288] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eeb817af5aba45e28e8f02b725b17a78 (127.11.192.254:38651), 85d78ecf62004006823ed80b0e7177ce (127.11.192.252:34747)
I20260812 07:05:36.411865 4242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca5c9e038aee4b2397d27e372ac57214" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eeb817af5aba45e28e8f02b725b17a78" is_pre_election: true
I20260812 07:05:36.411975 4543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca5c9e038aee4b2397d27e372ac57214" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85d78ecf62004006823ed80b0e7177ce" is_pre_election: true
I20260812 07:05:36.412528 4242 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca5c9e038aee4b2397d27e372ac57214 in term 0.
I20260812 07:05:36.412549 4543 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca5c9e038aee4b2397d27e372ac57214 in term 0.
I20260812 07:05:36.413089 4332 leader_election.cc:302] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85d78ecf62004006823ed80b0e7177ce, ca5c9e038aee4b2397d27e372ac57214; no voters:
I20260812 07:05:36.413743 4677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:05:36.413898 4677 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:05:36.414001 4677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:05:36.417550 4677 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb817af5aba45e28e8f02b725b17a78" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 } } peers { permanent_uuid: "ca5c9e038aee4b2397d27e372ac57214" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } } peers { permanent_uuid: "85d78ecf62004006823ed80b0e7177ce" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } }
I20260812 07:05:36.418704 4242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca5c9e038aee4b2397d27e372ac57214" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eeb817af5aba45e28e8f02b725b17a78"
I20260812 07:05:36.418936 4242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:05:36.422384 4242 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5c9e038aee4b2397d27e372ac57214 in term 1.
I20260812 07:05:36.426934 4334 leader_election.cc:302] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca5c9e038aee4b2397d27e372ac57214, eeb817af5aba45e28e8f02b725b17a78; no voters:
I20260812 07:05:36.429787 4543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca5c9e038aee4b2397d27e372ac57214" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85d78ecf62004006823ed80b0e7177ce"
I20260812 07:05:36.430022 4543 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:05:36.433374 4543 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5c9e038aee4b2397d27e372ac57214 in term 1.
I20260812 07:05:36.434018 4677 leader_election.cc:288] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [CANDIDATE]: Term 1 election: Requested vote from peers eeb817af5aba45e28e8f02b725b17a78 (127.11.192.254:38651), 85d78ecf62004006823ed80b0e7177ce (127.11.192.252:34747)
I20260812 07:05:36.434232 4677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:05:36.434376 4677 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [term 1 LEADER]: Becoming Leader. State: Replica: ca5c9e038aee4b2397d27e372ac57214, State: Running, Role: LEADER
I20260812 07:05:36.434963 4677 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb817af5aba45e28e8f02b725b17a78" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 } } peers { permanent_uuid: "ca5c9e038aee4b2397d27e372ac57214" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } } peers { permanent_uuid: "85d78ecf62004006823ed80b0e7177ce" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } }
I20260812 07:05:36.455837 4697 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca5c9e038aee4b2397d27e372ac57214. Latest consensus state: current_term: 1 leader_uuid: "ca5c9e038aee4b2397d27e372ac57214" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb817af5aba45e28e8f02b725b17a78" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 } } peers { permanent_uuid: "ca5c9e038aee4b2397d27e372ac57214" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } } peers { permanent_uuid: "85d78ecf62004006823ed80b0e7177ce" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } } }
I20260812 07:05:36.456295 4697 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [sys.catalog]: This master's current role is: LEADER
I20260812 07:05:36.463119 4701 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:05:36.465338 4701 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:05:36.484867 4543 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce [term 1 FOLLOWER]: Refusing update from remote peer ca5c9e038aee4b2397d27e372ac57214: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:05:36.485872 4697 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85d78ecf62004006823ed80b0e7177ce" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:05:36.493002 4242 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78 [term 1 FOLLOWER]: Refusing update from remote peer ca5c9e038aee4b2397d27e372ac57214: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:05:36.493757 4697 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eeb817af5aba45e28e8f02b725b17a78" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:05:36.524228 4617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca5c9e038aee4b2397d27e372ac57214. Latest consensus state: current_term: 1 leader_uuid: "ca5c9e038aee4b2397d27e372ac57214" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb817af5aba45e28e8f02b725b17a78" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 } } peers { permanent_uuid: "ca5c9e038aee4b2397d27e372ac57214" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } } peers { permanent_uuid: "85d78ecf62004006823ed80b0e7177ce" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } } }
I20260812 07:05:36.524566 4617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:05:36.539804 4703 mvcc.cc:205] Tried to move back new op lower bound from 7317579106230403072 to 7317579106052038656. Current Snapshot: MvccSnapshot[applied={T|T < 7317579106230403072}]
I20260812 07:05:36.551111 4654 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca5c9e038aee4b2397d27e372ac57214. Latest consensus state: current_term: 1 leader_uuid: "ca5c9e038aee4b2397d27e372ac57214" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb817af5aba45e28e8f02b725b17a78" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 } } peers { permanent_uuid: "ca5c9e038aee4b2397d27e372ac57214" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } } peers { permanent_uuid: "85d78ecf62004006823ed80b0e7177ce" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } } }
I20260812 07:05:36.551409 4654 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:05:36.557122 4654 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca5c9e038aee4b2397d27e372ac57214" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb817af5aba45e28e8f02b725b17a78" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 } } peers { permanent_uuid: "ca5c9e038aee4b2397d27e372ac57214" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } } peers { permanent_uuid: "85d78ecf62004006823ed80b0e7177ce" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } } }
I20260812 07:05:36.557374 4654 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:05:36.558173 4710 mvcc.cc:205] Tried to move back new op lower bound from 7317579106230403072 to 7317579106052038656. Current Snapshot: MvccSnapshot[applied={T|T < 7317579106230403072}]
I20260812 07:05:36.565639 4677 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca5c9e038aee4b2397d27e372ac57214" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb817af5aba45e28e8f02b725b17a78" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 } } peers { permanent_uuid: "ca5c9e038aee4b2397d27e372ac57214" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } } peers { permanent_uuid: "85d78ecf62004006823ed80b0e7177ce" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } } }
I20260812 07:05:36.566016 4677 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [sys.catalog]: This master's current role is: LEADER
I20260812 07:05:36.566426 4677 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca5c9e038aee4b2397d27e372ac57214" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb817af5aba45e28e8f02b725b17a78" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 } } peers { permanent_uuid: "ca5c9e038aee4b2397d27e372ac57214" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } } peers { permanent_uuid: "85d78ecf62004006823ed80b0e7177ce" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } } }
I20260812 07:05:36.566694 4677 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214 [sys.catalog]: This master's current role is: LEADER
I20260812 07:05:36.569780 4701 catalog_manager.cc:1347] Generated new cluster ID: 099b26a83a604c79aa603e662fe12c6b
I20260812 07:05:36.569885 4701 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:05:36.586925 4617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca5c9e038aee4b2397d27e372ac57214" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb817af5aba45e28e8f02b725b17a78" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 38651 } } peers { permanent_uuid: "ca5c9e038aee4b2397d27e372ac57214" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 41723 } } peers { permanent_uuid: "85d78ecf62004006823ed80b0e7177ce" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 34747 } } }
I20260812 07:05:36.587200 4617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:05:36.660610 4701 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:05:36.662500 4701 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:05:36.684362 4701 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca5c9e038aee4b2397d27e372ac57214: Generated new TSK 0
I20260812 07:05:36.685386 4701 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:05:37.236765 4644 catalog_manager.cc:1259] Loaded cluster ID: 099b26a83a604c79aa603e662fe12c6b
I20260812 07:05:37.236864 4644 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce: loading cluster ID for follower catalog manager: success
I20260812 07:05:37.239284 4644 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce: acquiring CA information for follower catalog manager: success
I20260812 07:05:37.241022 4644 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 85d78ecf62004006823ed80b0e7177ce: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 07:05:37.328207 4671 catalog_manager.cc:1259] Loaded cluster ID: 099b26a83a604c79aa603e662fe12c6b
I20260812 07:05:37.328294 4671 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78: loading cluster ID for follower catalog manager: success
I20260812 07:05:37.330789 4671 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78: acquiring CA information for follower catalog manager: success
I20260812 07:05:37.332576 4671 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eeb817af5aba45e28e8f02b725b17a78: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 07:05:37.810420 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 4172
I20260812 07:05:37.811683 4281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:05:37.919590 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 4172
I20260812 07:05:37.950269 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 4307
I20260812 07:05:37.951387 4423 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:05:38.057962 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 4307
I20260812 07:05:38.082628 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 4441
I20260812 07:05:38.083696 4575 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:05:38.190047 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 4441
2026-08-12T07:05:38Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4955 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:05:38 dist-test-slave-lgr5 krb5kdc[4746](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:05:38 dist-test-slave-lgr5 krb5kdc[4746](info): set up 2 sockets
Aug 12 07:05:38 dist-test-slave-lgr5 krb5kdc[4746](info): commencing operation
krb5kdc: starting...
W20260812 07:05:39.438246 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.007s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:05:39 dist-test-slave-lgr5 krb5kdc[4746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:05:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:05:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:05:39.671088 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:40305
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:37619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:40305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:05:39.997360 4767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:39.997731 4767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:39.997915 4767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:40.006834 4767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:05:40.006959 4767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:40.007081 4767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:40.007160 4767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:05:40.007222 4767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:05:40.017831 4767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:37619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:40305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:40305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:40.019133 4767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:40.020506 4767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:05:40.031755 4773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:40.031826 4772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:40.031792 4779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:40.032612 4767 server_base.cc:1034] running on GCE node
I20260812 07:05:40.033303 4767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:40.034601 4767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:40.035794 4767 hybrid_clock.cc:648] HybridClock initialized: now 1786518340035753 us; error 31 us; skew 500 ppm
I20260812 07:05:40.036154 4767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:40 dist-test-slave-lgr5 krb5kdc[4746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518340, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:40.039070 4767 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:05:40.040452 4767 webserver.cc:460] Webserver started at http://127.11.192.254:33631/ using document root <none> and password file <none>
I20260812 07:05:40.041095 4767 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:40.041204 4767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:05:40.041508 4767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:05:40.044390 4767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "ff0c9c00d9e24282ba5961a30cbf039e"
format_stamp: "Formatted at 2026-08-12 07:05:40 on dist-test-slave-lgr5"
server_key: "b0c984e4999b2dfb9a8d98ee8370e25d"
server_key_iv: "70b9f9179332e49b9188483effcb5b18"
server_key_version: "encryptionkey@0"
I20260812 07:05:40.045125 4767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "ff0c9c00d9e24282ba5961a30cbf039e"
format_stamp: "Formatted at 2026-08-12 07:05:40 on dist-test-slave-lgr5"
server_key: "b0c984e4999b2dfb9a8d98ee8370e25d"
server_key_iv: "70b9f9179332e49b9188483effcb5b18"
server_key_version: "encryptionkey@0"
I20260812 07:05:40.049962 4767 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 07:05:40.053285 4795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:40.054539 4767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:05:40.054700 4767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "ff0c9c00d9e24282ba5961a30cbf039e"
format_stamp: "Formatted at 2026-08-12 07:05:40 on dist-test-slave-lgr5"
server_key: "b0c984e4999b2dfb9a8d98ee8370e25d"
server_key_iv: "70b9f9179332e49b9188483effcb5b18"
server_key_version: "encryptionkey@0"
I20260812 07:05:40.054869 4767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:40.073417 4767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:40.077180 4767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:40.077430 4767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:40.094301 4767 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:40305
I20260812 07:05:40.094331 4903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:40305 every 8 connection(s)
I20260812 07:05:40.095597 4767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:05:40.099109 4906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:05:40.103951 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 4767
I20260812 07:05:40.104188 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:05:40.104879 12035 external_mini_cluster.cc:1428] Setting key 9ae3aeceb3b107d1b0a7b2c4a95ac877
I20260812 07:05:40.110299 4906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e: Bootstrap starting.
I20260812 07:05:40.113785 4906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e: Neither blocks nor log segments found. Creating new log.
I20260812 07:05:40.114813 4906 log.cc:824] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e: Log is configured to *not* fsync() on all Append() calls
I20260812 07:05:40.117458 4906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e: No bootstrap required, opened a new log
Aug 12 07:05:40 dist-test-slave-lgr5 krb5kdc[4746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:05:40.124825 4906 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0c9c00d9e24282ba5961a30cbf039e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40305 } }
I20260812 07:05:40.125144 4906 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:05:40.125244 4906 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff0c9c00d9e24282ba5961a30cbf039e, State: Initialized, Role: FOLLOWER
I20260812 07:05:40.125789 4906 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0c9c00d9e24282ba5961a30cbf039e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40305 } }
I20260812 07:05:40.126066 4906 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:05:40.126199 4906 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:05:40.126370 4906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:05:40.127110 4915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:40.106596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47752 (local address 127.11.192.254:40305)
0812 07:05:40.107485 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:40.107504 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:40.107552 (+ 48us) server_negotiation.cc:407] Connection header received
0812 07:05:40.109859 (+ 2307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:40.109887 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:40.110343 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:40.110748 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:40.112063 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:40.113960 (+ 1897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:40.115323 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:40.116013 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:40.116220 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:40.120019 (+ 3799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:40.120027 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:40.120043 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:40.120116 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:40.123115 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:40.123796 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:40.123802 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:40.123807 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:40.123984 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:40.124392 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:40.124396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:40.124398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:40.124892 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:40.125169 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:40.125681 (+ 512us) server_negotiation.cc:299] Negotiation successful
0812 07:05:40.126070 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":365,"threads_started":1}
I20260812 07:05:40.128782 4906 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0c9c00d9e24282ba5961a30cbf039e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40305 } }
I20260812 07:05:40.129274 4906 leader_election.cc:302] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff0c9c00d9e24282ba5961a30cbf039e; no voters:
I20260812 07:05:40.129827 4906 leader_election.cc:288] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:05:40.129944 4920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:05:40.131057 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:05:40.131006 4920 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e [term 1 LEADER]: Becoming Leader. State: Replica: ff0c9c00d9e24282ba5961a30cbf039e, State: Running, Role: LEADER
I20260812 07:05:40.131428 4920 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0c9c00d9e24282ba5961a30cbf039e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40305 } }
I20260812 07:05:40.131819 4906 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:05:40.135201 4921 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff0c9c00d9e24282ba5961a30cbf039e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0c9c00d9e24282ba5961a30cbf039e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40305 } } }
I20260812 07:05:40.135349 4922 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff0c9c00d9e24282ba5961a30cbf039e. Latest consensus state: current_term: 1 leader_uuid: "ff0c9c00d9e24282ba5961a30cbf039e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff0c9c00d9e24282ba5961a30cbf039e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40305 } } }
I20260812 07:05:40.135669 4921 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e [sys.catalog]: This master's current role is: LEADER
I20260812 07:05:40.135710 4922 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e [sys.catalog]: This master's current role is: LEADER
I20260812 07:05:40.142376 4936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:05:40.146275 4936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:05:40.154244 4936 catalog_manager.cc:1347] Generated new cluster ID: 5f5bc58b2cf24942aab409d074aeab63
I20260812 07:05:40.154340 4936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:05:40.178102 4936 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:05:40.179390 4936 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:05:40.187520 4936 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff0c9c00d9e24282ba5961a30cbf039e: Generated new TSK 0
I20260812 07:05:40.188366 4936 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:05:40.244014 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 4767
I20260812 07:05:40.245126 4892 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:05:40.339700 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 4767
2026-08-12T07:05:40Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2151 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:05:40 dist-test-slave-lgr5 krb5kdc[4960](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:05:40 dist-test-slave-lgr5 krb5kdc[4960](info): set up 2 sockets
Aug 12 07:05:40 dist-test-slave-lgr5 krb5kdc[4960](info): commencing operation
krb5kdc: starting...
W20260812 07:05:41.593715 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:05:41 dist-test-slave-lgr5 krb5kdc[4960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:05:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:05:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:05:41.821465 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:33305
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:33305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:05:42.143151 4982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:42.143431 4982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:42.143535 4982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:42.150708 4982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:05:42.150825 4982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:42.150885 4982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:42.150949 4982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:05:42.150998 4982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:05:42.160660 4982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:33305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:33305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:42.161809 4982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:42.163064 4982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:05:42.174391 4991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:42.174430 4996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:42.174465 4993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:42.175422 4982 server_base.cc:1034] running on GCE node
I20260812 07:05:42.176198 4982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:42.177510 4982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:42.178740 4982 hybrid_clock.cc:648] HybridClock initialized: now 1786518342178696 us; error 28 us; skew 500 ppm
I20260812 07:05:42.179095 4982 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:42 dist-test-slave-lgr5 krb5kdc[4960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518342, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:42.185573 4982 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:05:42.187036 4982 webserver.cc:460] Webserver started at http://127.11.192.254:46083/ using document root <none> and password file <none>
I20260812 07:05:42.187709 4982 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:42.187819 4982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:05:42.188122 4982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:05:42.191195 4982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "6ac28bd0e3db4a459678953e93aa9b57"
format_stamp: "Formatted at 2026-08-12 07:05:42 on dist-test-slave-lgr5"
server_key: "bfff20c1ef62407c83002a76b6d33ca0"
server_key_iv: "1994d35f02b725364d8cd1fd417cae8c"
server_key_version: "encryptionkey@0"
I20260812 07:05:42.191970 4982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "6ac28bd0e3db4a459678953e93aa9b57"
format_stamp: "Formatted at 2026-08-12 07:05:42 on dist-test-slave-lgr5"
server_key: "bfff20c1ef62407c83002a76b6d33ca0"
server_key_iv: "1994d35f02b725364d8cd1fd417cae8c"
server_key_version: "encryptionkey@0"
I20260812 07:05:42.197149 4982 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 07:05:42.200528 5008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:42.201781 4982 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 07:05:42.201946 4982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "6ac28bd0e3db4a459678953e93aa9b57"
format_stamp: "Formatted at 2026-08-12 07:05:42 on dist-test-slave-lgr5"
server_key: "bfff20c1ef62407c83002a76b6d33ca0"
server_key_iv: "1994d35f02b725364d8cd1fd417cae8c"
server_key_version: "encryptionkey@0"
I20260812 07:05:42.202137 4982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:42.225505 4982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:42.229214 4982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:42.229473 4982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:42.246348 4982 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:33305
I20260812 07:05:42.246395 5110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:33305 every 8 connection(s)
I20260812 07:05:42.247582 4982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:05:42.251004 5111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:05:42.253010 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 4982
I20260812 07:05:42.253242 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:05:42.254062 12035 external_mini_cluster.cc:1428] Setting key 95d50aebc5486a56a92a005c9cf9168a
I20260812 07:05:42.262928 5111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57: Bootstrap starting.
I20260812 07:05:42.266461 5111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57: Neither blocks nor log segments found. Creating new log.
I20260812 07:05:42.267715 5111 log.cc:824] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57: Log is configured to *not* fsync() on all Append() calls
Aug 12 07:05:42 dist-test-slave-lgr5 krb5kdc[4960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:05:42.270661 5111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57: No bootstrap required, opened a new log
I20260812 07:05:42.276799 5116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:42.256113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36328 (local address 127.11.192.254:33305)
0812 07:05:42.256883 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:42.256894 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:42.256938 (+ 44us) server_negotiation.cc:407] Connection header received
0812 07:05:42.258618 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:42.258640 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:42.258988 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:42.259282 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:42.261061 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:42.262302 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:42.264121 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:42.264827 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:42.264955 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:42.269796 (+ 4841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:42.269803 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:42.269816 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:42.269857 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:42.272812 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:42.273593 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:42.273599 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:42.273604 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:42.273758 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:42.274217 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:42.274220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:42.274225 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:42.274716 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:42.274950 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:42.275425 (+ 475us) server_negotiation.cc:299] Negotiation successful
0812 07:05:42.275753 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":347,"threads_started":1}
I20260812 07:05:42.278929 5111 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac28bd0e3db4a459678953e93aa9b57" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33305 } }
I20260812 07:05:42.279273 5111 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:05:42.279379 5111 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ac28bd0e3db4a459678953e93aa9b57, State: Initialized, Role: FOLLOWER
I20260812 07:05:42.279930 5111 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac28bd0e3db4a459678953e93aa9b57" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33305 } }
I20260812 07:05:42.280160 5111 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:05:42.280284 5111 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:05:42.280431 5111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:05:42.282379 5111 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac28bd0e3db4a459678953e93aa9b57" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33305 } }
I20260812 07:05:42.282833 5111 leader_election.cc:302] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ac28bd0e3db4a459678953e93aa9b57; no voters:
I20260812 07:05:42.283434 5111 leader_election.cc:288] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:05:42.283550 5120 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:05:42.284231 5120 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 1 LEADER]: Becoming Leader. State: Replica: 6ac28bd0e3db4a459678953e93aa9b57, State: Running, Role: LEADER
I20260812 07:05:42.284732 5120 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac28bd0e3db4a459678953e93aa9b57" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33305 } }
I20260812 07:05:42.285295 5111 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:05:42.287415 5121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ac28bd0e3db4a459678953e93aa9b57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac28bd0e3db4a459678953e93aa9b57" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33305 } } }
I20260812 07:05:42.287771 5121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [sys.catalog]: This master's current role is: LEADER
I20260812 07:05:42.288473 5122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ac28bd0e3db4a459678953e93aa9b57. Latest consensus state: current_term: 1 leader_uuid: "6ac28bd0e3db4a459678953e93aa9b57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac28bd0e3db4a459678953e93aa9b57" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33305 } } }
I20260812 07:05:42.288830 5122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [sys.catalog]: This master's current role is: LEADER
I20260812 07:05:42.289242 5132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:05:42.294381 5132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:05:42.302156 5132 catalog_manager.cc:1347] Generated new cluster ID: 9bd82972dc7b4fe39bbd376884eb3d01
I20260812 07:05:42.302240 5132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:05:42.341787 5132 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:05:42.343604 5132 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:05:42.353549 5132 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57: Generated new TSK 0
I20260812 07:05:42.354280 5132 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
I20260812 07:05:42.435231 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:33305
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:05:42.788185 5153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:42.788431 5153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:42.788568 5153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:42.795542 5153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:42.795642 5153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:42.795796 5153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:05:42.805299 5153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:33305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:42.806437 5153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:42.807646 5153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:05:42.819828 5172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:42.819784 5168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:42.819872 5169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:42.821039 5153 server_base.cc:1034] running on GCE node
I20260812 07:05:42.821597 5153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:42.822455 5153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:42.823654 5153 hybrid_clock.cc:648] HybridClock initialized: now 1786518342823612 us; error 31 us; skew 500 ppm
I20260812 07:05:42.823938 5153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:42 dist-test-slave-lgr5 krb5kdc[4960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518342, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:42.826974 5153 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:05:42.828277 5153 webserver.cc:460] Webserver started at http://127.11.192.193:43697/ using document root <none> and password file <none>
I20260812 07:05:42.828857 5153 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:42.828953 5153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:05:42.829191 5153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:05:42.831871 5153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "6808c4ba75bb4745bb6bafada929a5ff"
format_stamp: "Formatted at 2026-08-12 07:05:42 on dist-test-slave-lgr5"
server_key: "e16011c21a5c65ac0ad160fff8b99161"
server_key_iv: "f4ddf39f064812a829e49f7401e36b35"
server_key_version: "encryptionkey@0"
I20260812 07:05:42.832623 5153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "6808c4ba75bb4745bb6bafada929a5ff"
format_stamp: "Formatted at 2026-08-12 07:05:42 on dist-test-slave-lgr5"
server_key: "e16011c21a5c65ac0ad160fff8b99161"
server_key_iv: "f4ddf39f064812a829e49f7401e36b35"
server_key_version: "encryptionkey@0"
I20260812 07:05:42.837285 5153 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260812 07:05:42.840425 5184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:42.841756 5153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 07:05:42.841912 5153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "6808c4ba75bb4745bb6bafada929a5ff"
format_stamp: "Formatted at 2026-08-12 07:05:42 on dist-test-slave-lgr5"
server_key: "e16011c21a5c65ac0ad160fff8b99161"
server_key_iv: "f4ddf39f064812a829e49f7401e36b35"
server_key_version: "encryptionkey@0"
I20260812 07:05:42.842104 5153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:42.855232 5153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:42.858765 5153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:42.859053 5153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:42.859925 5153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:05:42.861186 5153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:05:42.861261 5153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:42.861374 5153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:05:42.861430 5153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:42.890929 5153 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:43141
I20260812 07:05:42.890985 5411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:43141 every 8 connection(s)
I20260812 07:05:42.892154 5153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:05:42.902426 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 5153
I20260812 07:05:42.902666 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:05:42.903518 12035 external_mini_cluster.cc:1428] Setting key cb4a3be830764f8620fb4ad5d293bb4b
Aug 12 07:05:42 dist-test-slave-lgr5 krb5kdc[4960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518342, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:05:42.911355 5414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:42.894518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:58935 (local address 127.11.192.254:33305)
0812 07:05:42.894941 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:42.894945 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:42.895926 (+ 981us) server_negotiation.cc:407] Connection header received
0812 07:05:42.897708 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:42.897712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:42.897779 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:42.897875 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:42.899249 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:42.899969 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:42.901448 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:42.901834 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:42.901926 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:42.905949 (+ 4023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:42.905954 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:42.905958 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:42.905995 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:42.908482 (+ 2487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:42.909157 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:42.909161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:42.909164 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:42.909283 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:42.909710 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:42.909714 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:42.909716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:42.909961 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:42.910103 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:42.910786 (+ 683us) server_negotiation.cc:299] Negotiation successful
0812 07:05:42.911017 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":205,"threads_started":1}
I20260812 07:05:42.912245 5418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:42.895084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.193:58935)
0812 07:05:42.895763 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:42.895803 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:42.897458 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:42.898100 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:42.898108 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:42.898581 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:05:42.899121 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:42.899139 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:42.900126 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:42.900130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:42.901334 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:42.901347 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:42.902078 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:42.902088 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:42.902371 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:42.903052 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:05:42.903081 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:05:42.905736 (+ 2655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:05:42.908620 (+ 2884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:42.908629 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:42.908642 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:42.909029 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:42.909390 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:42.909395 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:42.909399 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:42.909591 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:42.910101 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:05:42.910108 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:05:42.910408 (+ 300us) client_negotiation.cc:769] Sending connection context
0812 07:05:42.910633 (+ 225us) client_negotiation.cc:240] Negotiation successful
0812 07:05:42.911099 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":256,"threads_started":1}
I20260812 07:05:42.913667 5412 heartbeater.cc:347] Connected to a master server at 127.11.192.254:33305
I20260812 07:05:42.913926 5412 heartbeater.cc:464] Registering TS with master...
I20260812 07:05:42.914613 5412 heartbeater.cc:511] Master 127.11.192.254:33305 requested a full tablet report, sending...
I20260812 07:05:42.916631 5037 ts_manager.cc:194] Registered new tserver with Master: 6808c4ba75bb4745bb6bafada929a5ff (127.11.192.193:43141)
I20260812 07:05:42.917996 5037 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:58935
WARNING: no policy specified for kudu/127.11.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
I20260812 07:05:42.957229 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:33305
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:05:43.288213 5422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:43.288520 5422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:43.288673 5422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:43.296033 5422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:43.296164 5422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:43.296336 5422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:05:43.305881 5422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:33305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:43.307065 5422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:43.308390 5422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:05:43.321455 5438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:43.321426 5436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:43.321965 5434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:43.322419 5422 server_base.cc:1034] running on GCE node
I20260812 07:05:43.322983 5422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:43.323841 5422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:43.325040 5422 hybrid_clock.cc:648] HybridClock initialized: now 1786518343324920 us; error 112 us; skew 500 ppm
I20260812 07:05:43.325326 5422 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:43 dist-test-slave-lgr5 krb5kdc[4960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:43.328411 5422 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:05:43.329737 5422 webserver.cc:460] Webserver started at http://127.11.192.194:36067/ using document root <none> and password file <none>
I20260812 07:05:43.330355 5422 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:43.330472 5422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:05:43.330747 5422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:05:43.333505 5422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "8c98955033894142b2ce660be9f2417d"
format_stamp: "Formatted at 2026-08-12 07:05:43 on dist-test-slave-lgr5"
server_key: "5a0750f971e6f0e23c4fdf37a6acabfb"
server_key_iv: "6e0e64a84750a0738841d3ff384ab084"
server_key_version: "encryptionkey@0"
I20260812 07:05:43.334251 5422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "8c98955033894142b2ce660be9f2417d"
format_stamp: "Formatted at 2026-08-12 07:05:43 on dist-test-slave-lgr5"
server_key: "5a0750f971e6f0e23c4fdf37a6acabfb"
server_key_iv: "6e0e64a84750a0738841d3ff384ab084"
server_key_version: "encryptionkey@0"
I20260812 07:05:43.339021 5422 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260812 07:05:43.342238 5454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:43.343463 5422 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 07:05:43.343616 5422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "8c98955033894142b2ce660be9f2417d"
format_stamp: "Formatted at 2026-08-12 07:05:43 on dist-test-slave-lgr5"
server_key: "5a0750f971e6f0e23c4fdf37a6acabfb"
server_key_iv: "6e0e64a84750a0738841d3ff384ab084"
server_key_version: "encryptionkey@0"
I20260812 07:05:43.343799 5422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:43.373948 5422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:43.377549 5422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:43.377799 5422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:43.378715 5422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:05:43.380008 5422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:05:43.380106 5422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:43.380231 5422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:05:43.380295 5422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:43.411242 5422 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:45927
I20260812 07:05:43.411286 5673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:45927 every 8 connection(s)
I20260812 07:05:43.412578 5422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:05:43.419500 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 5422
I20260812 07:05:43.419656 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:05:43.420099 12035 external_mini_cluster.cc:1428] Setting key 702d7ad35bccdac81665f51d8c8681d1
Aug 12 07:05:43 dist-test-slave-lgr5 krb5kdc[4960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:05:43.431718 5676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:43.414987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:48673 (local address 127.11.192.254:33305)
0812 07:05:43.415426 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:43.415432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:43.416436 (+ 1004us) server_negotiation.cc:407] Connection header received
0812 07:05:43.418124 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:43.418129 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:43.418194 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:43.418285 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:43.419742 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:43.420557 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:43.422344 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:43.422760 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:43.422870 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:43.426347 (+ 3477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:43.426352 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:43.426355 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:43.426388 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:43.428913 (+ 2525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:43.429679 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:43.429683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:43.429685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:43.429805 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:43.430231 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:43.430234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:43.430236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:43.430459 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:43.430571 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:43.431297 (+ 726us) server_negotiation.cc:299] Negotiation successful
0812 07:05:43.431457 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":208,"threads_started":1}
I20260812 07:05:43.432881 5677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:43.415580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.194:48673)
0812 07:05:43.416284 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:43.416325 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:43.417885 (+ 1560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:43.418472 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:43.418482 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:43.419006 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:05:43.419549 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:43.419572 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:43.420711 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:43.420716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:43.422200 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:43.422213 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:43.422989 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:43.422998 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:43.423201 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:43.423811 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:05:43.423832 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:05:43.426118 (+ 2286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:05:43.429050 (+ 2932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:43.429059 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:43.429072 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:43.429580 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:43.429899 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:43.429903 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:43.429906 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:43.430142 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:43.430584 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:05:43.430592 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:05:43.430954 (+ 362us) client_negotiation.cc:769] Sending connection context
0812 07:05:43.431237 (+ 283us) client_negotiation.cc:240] Negotiation successful
0812 07:05:43.431690 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":268,"threads_started":1}
I20260812 07:05:43.434151 5674 heartbeater.cc:347] Connected to a master server at 127.11.192.254:33305
I20260812 07:05:43.434458 5674 heartbeater.cc:464] Registering TS with master...
I20260812 07:05:43.435223 5674 heartbeater.cc:511] Master 127.11.192.254:33305 requested a full tablet report, sending...
I20260812 07:05:43.436591 5037 ts_manager.cc:194] Registered new tserver with Master: 8c98955033894142b2ce660be9f2417d (127.11.192.194:45927)
I20260812 07:05:43.437628 5037 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:48673
WARNING: no policy specified for kudu/127.11.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
I20260812 07:05:43.477260 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:33305
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:05:43.797721 5682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:43.798012 5682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:43.798170 5682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:43.805097 5682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:43.805208 5682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:43.805363 5682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:05:43.814924 5682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:33305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:43.816160 5682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:43.817441 5682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:05:43.829612 5694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:43.829797 5695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:43.830312 5697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:43.830729 5682 server_base.cc:1034] running on GCE node
I20260812 07:05:43.831288 5682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:43.832139 5682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:43.833348 5682 hybrid_clock.cc:648] HybridClock initialized: now 1786518343833289 us; error 51 us; skew 500 ppm
I20260812 07:05:43.833634 5682 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:43 dist-test-slave-lgr5 krb5kdc[4960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:43.836606 5682 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:05:43.837867 5682 webserver.cc:460] Webserver started at http://127.11.192.195:46317/ using document root <none> and password file <none>
I20260812 07:05:43.838500 5682 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:43.838609 5682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:05:43.838892 5682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:05:43.841652 5682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "8db96391b7384bbf983337c330332568"
format_stamp: "Formatted at 2026-08-12 07:05:43 on dist-test-slave-lgr5"
server_key: "1ec9dfbf108e524251d730a2bd9ca28b"
server_key_iv: "b4bedc7c48b21f015a77a8c4fad05a27"
server_key_version: "encryptionkey@0"
I20260812 07:05:43.842389 5682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "8db96391b7384bbf983337c330332568"
format_stamp: "Formatted at 2026-08-12 07:05:43 on dist-test-slave-lgr5"
server_key: "1ec9dfbf108e524251d730a2bd9ca28b"
server_key_iv: "b4bedc7c48b21f015a77a8c4fad05a27"
server_key_version: "encryptionkey@0"
I20260812 07:05:43.847373 5682 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260812 07:05:43.850725 5713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:43.851958 5682 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:05:43.852154 5682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "8db96391b7384bbf983337c330332568"
format_stamp: "Formatted at 2026-08-12 07:05:43 on dist-test-slave-lgr5"
server_key: "1ec9dfbf108e524251d730a2bd9ca28b"
server_key_iv: "b4bedc7c48b21f015a77a8c4fad05a27"
server_key_version: "encryptionkey@0"
I20260812 07:05:43.852384 5682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:43.864013 5682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:43.867437 5682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:43.867664 5682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:43.868501 5682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:05:43.869674 5682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:05:43.869748 5682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:43.869831 5682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:05:43.869874 5682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:43.901336 5682 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:33705
I20260812 07:05:43.901369 5920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:33705 every 8 connection(s)
I20260812 07:05:43.902478 5682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:05:43.909907 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 5682
I20260812 07:05:43.910072 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:05:43.910501 12035 external_mini_cluster.cc:1428] Setting key 34e3f5953aa478687bfd1a8897b688a1
Aug 12 07:05:43 dist-test-slave-lgr5 krb5kdc[4960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:05:43.920658 5412 heartbeater.cc:503] Master 127.11.192.254:33305 was elected leader, sending a full tablet report...
I20260812 07:05:43.921049 5676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:43.904905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:37967 (local address 127.11.192.254:33305)
0812 07:05:43.905087 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:43.905091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:43.906158 (+ 1067us) server_negotiation.cc:407] Connection header received
0812 07:05:43.907975 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:43.907981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:43.908060 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:43.908182 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:43.909597 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:43.910395 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:43.912136 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:43.912665 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:43.912776 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:43.915961 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:43.915966 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:43.915969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:43.916003 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:43.918264 (+ 2261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:43.918901 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:43.918907 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:43.918910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:43.919031 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:43.919433 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:43.919438 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:43.919441 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:43.919696 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:43.919840 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:43.920524 (+ 684us) server_negotiation.cc:299] Negotiation successful
0812 07:05:43.920748 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 07:05:43.921901 5929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:43.905294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.195:37967)
0812 07:05:43.905984 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:43.906056 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:43.907714 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:43.908421 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:43.908429 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:43.908892 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:05:43.909423 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:43.909441 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:43.910545 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:43.910549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:43.911971 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:43.911983 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:43.912880 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:43.912887 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:43.913155 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:43.913729 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:05:43.913749 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:05:43.915770 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:05:43.918395 (+ 2625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:43.918403 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:43.918416 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:43.918787 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:43.919147 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:43.919151 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:43.919153 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:43.919320 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:43.919837 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:05:43.919844 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:05:43.920163 (+ 319us) client_negotiation.cc:769] Sending connection context
0812 07:05:43.920391 (+ 228us) client_negotiation.cc:240] Negotiation successful
0812 07:05:43.920842 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":276,"threads_started":1}
I20260812 07:05:43.923215 5925 heartbeater.cc:347] Connected to a master server at 127.11.192.254:33305
I20260812 07:05:43.923465 5925 heartbeater.cc:464] Registering TS with master...
I20260812 07:05:43.924297 5925 heartbeater.cc:511] Master 127.11.192.254:33305 requested a full tablet report, sending...
I20260812 07:05:43.925787 5037 ts_manager.cc:194] Registered new tserver with Master: 8db96391b7384bbf983337c330332568 (127.11.192.195:33705)
I20260812 07:05:43.926570 5037 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:37967
I20260812 07:05:43.937503 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:05:43.953307 5676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:43.941756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36336 (local address 127.11.192.254:33305)
0812 07:05:43.941928 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:43.941934 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:43.942335 (+ 401us) server_negotiation.cc:407] Connection header received
0812 07:05:43.942463 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:43.942467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:43.942533 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:43.942608 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:43.943967 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:43.944697 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:43.946047 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:43.946501 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:43.946592 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:43.948924 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:43.948933 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:43.948936 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:43.948963 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:43.951168 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:43.951736 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:43.951739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:43.951741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:43.951843 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:43.952157 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:43.952160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:43.952162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:43.952445 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:43.952639 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:43.952864 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 07:05:43.953011 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:05:43.956959 5037 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:05:43.963557 5037 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:05:43.987396 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:43.975151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:33705 (local address 127.0.0.1:55298)
0812 07:05:43.977130 (+ 1979us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:43.977194 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:43.977407 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:43.979293 (+ 1886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:43.979300 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:43.979327 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:05:43.979881 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:43.979925 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:43.981980 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:43.981987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:43.984462 (+ 2475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:43.984480 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:43.986348 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:43.986355 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:43.986658 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:43.986718 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 07:05:43.986853 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 07:05:43.987043 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1631,"thread_start_us":2580,"threads_started":1}
I20260812 07:05:43.987403 5946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:43.975719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55298 (local address 127.11.192.195:33705)
0812 07:05:43.978754 (+ 3035us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:43.978762 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:43.978793 (+ 31us) server_negotiation.cc:407] Connection header received
0812 07:05:43.978882 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:43.978889 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:43.979096 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:43.979289 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:05:43.980213 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:43.981821 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:43.984658 (+ 2837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:43.986131 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:43.986334 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:43.986530 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:43.986901 (+ 371us) server_negotiation.cc:299] Negotiation successful
0812 07:05:43.987072 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2860,"thread_start_us":2673,"threads_started":1}
I20260812 07:05:43.991910 5796 tablet_service.cc:1467] Processing CreateTablet for tablet 9bc21f98b1da4cb9957e96f430e22171 (DEFAULT_TABLE table=test-table [id=50228a0c512244a9be85ed30f67e39a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:05:43.993453 5796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bc21f98b1da4cb9957e96f430e22171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:05:43.994964 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:43.975160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:43141 (local address 127.0.0.1:50348)
0812 07:05:43.977983 (+ 2823us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:43.978012 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:43.978120 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:43.987912 (+ 9792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:43.987918 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:43.987939 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:05:43.988358 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:43.988370 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:43.990648 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:43.990653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:43.992706 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:43.992722 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:43.994209 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:43.994215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:43.994380 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:43.994412 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 07:05:43.994488 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 07:05:43.994608 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2706,"mutex_wait_us":45,"thread_start_us":1471,"threads_started":1}
I20260812 07:05:43.995026 5951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:43.975809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50348 (local address 127.11.192.193:43141)
0812 07:05:43.987155 (+ 11346us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:43.987163 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:43.987191 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:05:43.987288 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:43.987295 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:43.987536 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:43.988581 (+ 1045us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:05:43.988850 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:43.990507 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:43.992906 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:43.994009 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:43.994200 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:43.994360 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:43.994574 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 07:05:43.994729 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11187,"thread_start_us":11872,"threads_started":1}
I20260812 07:05:43.995944 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:43.981217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:45927 (local address 127.0.0.1:41760)
0812 07:05:43.987768 (+ 6551us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:43.987784 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:43.987877 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:43.988571 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:43.988575 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:43.988600 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:05:43.988813 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:43.988825 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:43.991006 (+ 2181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:43.991012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:43.993500 (+ 2488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:43.993516 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:43.995266 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:43.995272 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:43.995432 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:43.995465 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:05:43.995535 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 07:05:43.995654 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6468,"mutex_wait_us":645,"thread_start_us":11178,"threads_started":1}
I20260812 07:05:43.997057 5953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:43.980907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41760 (local address 127.11.192.194:45927)
0812 07:05:43.981547 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:43.981556 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:43.988004 (+ 6448us) server_negotiation.cc:407] Connection header received
0812 07:05:43.988147 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:43.988156 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:43.988365 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:43.989023 (+ 658us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:05:43.989252 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:43.990829 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:43.993684 (+ 2855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:43.995077 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:43.996356 (+ 1279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:43.996511 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:43.996587 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 07:05:43.996741 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":485,"threads_started":1}
I20260812 07:05:43.998914 5271 tablet_service.cc:1467] Processing CreateTablet for tablet 9bc21f98b1da4cb9957e96f430e22171 (DEFAULT_TABLE table=test-table [id=50228a0c512244a9be85ed30f67e39a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:05:43.999662 5551 tablet_service.cc:1467] Processing CreateTablet for tablet 9bc21f98b1da4cb9957e96f430e22171 (DEFAULT_TABLE table=test-table [id=50228a0c512244a9be85ed30f67e39a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:05:44.000309 5271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bc21f98b1da4cb9957e96f430e22171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:05:44.000803 5551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bc21f98b1da4cb9957e96f430e22171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:05:44.010243 5960 tablet_bootstrap.cc:492] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568: Bootstrap starting.
I20260812 07:05:44.012120 5961 tablet_bootstrap.cc:492] T 9bc21f98b1da4cb9957e96f430e22171 P 8c98955033894142b2ce660be9f2417d: Bootstrap starting.
I20260812 07:05:44.013696 5960 tablet_bootstrap.cc:654] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568: Neither blocks nor log segments found. Creating new log.
I20260812 07:05:44.014317 5963 tablet_bootstrap.cc:492] T 9bc21f98b1da4cb9957e96f430e22171 P 6808c4ba75bb4745bb6bafada929a5ff: Bootstrap starting.
I20260812 07:05:44.015403 5960 log.cc:824] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568: Log is configured to *not* fsync() on all Append() calls
I20260812 07:05:44.015580 5961 tablet_bootstrap.cc:654] T 9bc21f98b1da4cb9957e96f430e22171 P 8c98955033894142b2ce660be9f2417d: Neither blocks nor log segments found. Creating new log.
I20260812 07:05:44.016776 5961 log.cc:824] T 9bc21f98b1da4cb9957e96f430e22171 P 8c98955033894142b2ce660be9f2417d: Log is configured to *not* fsync() on all Append() calls
I20260812 07:05:44.018054 5963 tablet_bootstrap.cc:654] T 9bc21f98b1da4cb9957e96f430e22171 P 6808c4ba75bb4745bb6bafada929a5ff: Neither blocks nor log segments found. Creating new log.
I20260812 07:05:44.018299 5960 tablet_bootstrap.cc:492] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568: No bootstrap required, opened a new log
I20260812 07:05:44.018575 5960 ts_tablet_manager.cc:1397] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 07:05:44.018992 5963 log.cc:824] T 9bc21f98b1da4cb9957e96f430e22171 P 6808c4ba75bb4745bb6bafada929a5ff: Log is configured to *not* fsync() on all Append() calls
I20260812 07:05:44.019716 5961 tablet_bootstrap.cc:492] T 9bc21f98b1da4cb9957e96f430e22171 P 8c98955033894142b2ce660be9f2417d: No bootstrap required, opened a new log
I20260812 07:05:44.019970 5961 ts_tablet_manager.cc:1397] T 9bc21f98b1da4cb9957e96f430e22171 P 8c98955033894142b2ce660be9f2417d: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260812 07:05:44.021296 5963 tablet_bootstrap.cc:492] T 9bc21f98b1da4cb9957e96f430e22171 P 6808c4ba75bb4745bb6bafada929a5ff: No bootstrap required, opened a new log
I20260812 07:05:44.021605 5963 ts_tablet_manager.cc:1397] T 9bc21f98b1da4cb9957e96f430e22171 P 6808c4ba75bb4745bb6bafada929a5ff: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260812 07:05:44.028779 5963 raft_consensus.cc:348] T 9bc21f98b1da4cb9957e96f430e22171 P 6808c4ba75bb4745bb6bafada929a5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8db96391b7384bbf983337c330332568" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33705 } } peers { permanent_uuid: "6808c4ba75bb4745bb6bafada929a5ff" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43141 } } peers { permanent_uuid: "8c98955033894142b2ce660be9f2417d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 45927 } }
I20260812 07:05:44.028752 5960 raft_consensus.cc:348] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8db96391b7384bbf983337c330332568" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33705 } } peers { permanent_uuid: "6808c4ba75bb4745bb6bafada929a5ff" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43141 } } peers { permanent_uuid: "8c98955033894142b2ce660be9f2417d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 45927 } }
I20260812 07:05:44.029119 5963 raft_consensus.cc:374] T 9bc21f98b1da4cb9957e96f430e22171 P 6808c4ba75bb4745bb6bafada929a5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:05:44.029143 5960 raft_consensus.cc:374] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:05:44.029230 5960 raft_consensus.cc:729] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8db96391b7384bbf983337c330332568, State: Initialized, Role: FOLLOWER
I20260812 07:05:44.029244 5963 raft_consensus.cc:729] T 9bc21f98b1da4cb9957e96f430e22171 P 6808c4ba75bb4745bb6bafada929a5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6808c4ba75bb4745bb6bafada929a5ff, State: Initialized, Role: FOLLOWER
I20260812 07:05:44.029789 5963 consensus_queue.cc:260] T 9bc21f98b1da4cb9957e96f430e22171 P 6808c4ba75bb4745bb6bafada929a5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8db96391b7384bbf983337c330332568" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33705 } } peers { permanent_uuid: "6808c4ba75bb4745bb6bafada929a5ff" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43141 } } peers { permanent_uuid: "8c98955033894142b2ce660be9f2417d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 45927 } }
I20260812 07:05:44.029915 5960 consensus_queue.cc:260] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8db96391b7384bbf983337c330332568" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33705 } } peers { permanent_uuid: "6808c4ba75bb4745bb6bafada929a5ff" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43141 } } peers { permanent_uuid: "8c98955033894142b2ce660be9f2417d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 45927 } }
I20260812 07:05:44.029825 5961 raft_consensus.cc:348] T 9bc21f98b1da4cb9957e96f430e22171 P 8c98955033894142b2ce660be9f2417d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8db96391b7384bbf983337c330332568" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33705 } } peers { permanent_uuid: "6808c4ba75bb4745bb6bafada929a5ff" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43141 } } peers { permanent_uuid: "8c98955033894142b2ce660be9f2417d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 45927 } }
I20260812 07:05:44.030210 5961 raft_consensus.cc:374] T 9bc21f98b1da4cb9957e96f430e22171 P 8c98955033894142b2ce660be9f2417d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:05:44.030349 5961 raft_consensus.cc:729] T 9bc21f98b1da4cb9957e96f430e22171 P 8c98955033894142b2ce660be9f2417d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c98955033894142b2ce660be9f2417d, State: Initialized, Role: FOLLOWER
I20260812 07:05:44.030956 5961 consensus_queue.cc:260] T 9bc21f98b1da4cb9957e96f430e22171 P 8c98955033894142b2ce660be9f2417d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8db96391b7384bbf983337c330332568" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33705 } } peers { permanent_uuid: "6808c4ba75bb4745bb6bafada929a5ff" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43141 } } peers { permanent_uuid: "8c98955033894142b2ce660be9f2417d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 45927 } }
I20260812 07:05:44.032256 5963 ts_tablet_manager.cc:1428] T 9bc21f98b1da4cb9957e96f430e22171 P 6808c4ba75bb4745bb6bafada929a5ff: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260812 07:05:44.033165 5925 heartbeater.cc:503] Master 127.11.192.254:33305 was elected leader, sending a full tablet report...
I20260812 07:05:44.033783 5960 ts_tablet_manager.cc:1428] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260812 07:05:44.036269 5674 heartbeater.cc:503] Master 127.11.192.254:33305 was elected leader, sending a full tablet report...
I20260812 07:05:44.036361 5961 ts_tablet_manager.cc:1428] T 9bc21f98b1da4cb9957e96f430e22171 P 8c98955033894142b2ce660be9f2417d: Time spent starting tablet: real 0.016s user 0.013s sys 0.005s
I20260812 07:05:44.126397 5970 raft_consensus.cc:482] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:05:44.126672 5970 raft_consensus.cc:504] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8db96391b7384bbf983337c330332568" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33705 } } peers { permanent_uuid: "6808c4ba75bb4745bb6bafada929a5ff" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43141 } } peers { permanent_uuid: "8c98955033894142b2ce660be9f2417d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 45927 } }
I20260812 07:05:44.128664 5970 leader_election.cc:288] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6808c4ba75bb4745bb6bafada929a5ff (127.11.192.193:43141), 8c98955033894142b2ce660be9f2417d (127.11.192.194:45927)
I20260812 07:05:44.135133 5929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:44.128870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:45927 (local address 127.11.192.195:55265)
0812 07:05:44.129172 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:44.129200 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:44.129335 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:44.129805 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:44.129810 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:44.129854 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:05:44.130081 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:44.130093 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:44.131331 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:44.131335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:44.133255 (+ 1920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:44.133274 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:44.134533 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:44.134538 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:44.134691 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:44.134720 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 07:05:44.134807 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 07:05:44.134899 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260812 07:05:44.135239 5953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:44.129095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:55265 (local address 127.11.192.194:45927)
0812 07:05:44.129346 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:44.129352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:44.129377 (+ 25us) server_negotiation.cc:407] Connection header received
0812 07:05:44.129457 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:44.129463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:44.129543 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:44.129633 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:05:44.130224 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:44.131172 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:44.133423 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:44.134381 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:44.134522 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:44.134595 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:44.134851 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 07:05:44.134977 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 07:05:44.135993 5974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:44.128992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:43141 (local address 127.11.192.195:34875)
0812 07:05:44.129927 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:44.129945 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:44.130077 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:44.130410 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:44.130414 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:44.130440 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:05:44.130732 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:44.130740 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:44.131884 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:44.131890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:44.133802 (+ 1912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:44.133817 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:44.135287 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:44.135294 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:44.135469 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:44.135506 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 07:05:44.135601 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 07:05:44.135691 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813,"mutex_wait_us":1,"thread_start_us":593,"threads_started":1}
I20260812 07:05:44.136269 5951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:44.129469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:34875 (local address 127.11.192.193:43141)
0812 07:05:44.129695 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:44.129701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:44.130035 (+ 334us) server_negotiation.cc:407] Connection header received
0812 07:05:44.130205 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:44.130210 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:44.130287 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:44.130383 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:05:44.130899 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:44.131723 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:44.133956 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:44.135073 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:44.135221 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:44.135289 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:44.135887 (+ 598us) server_negotiation.cc:299] Negotiation successful
0812 07:05:44.136016 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 07:05:44.136363 5594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bc21f98b1da4cb9957e96f430e22171" candidate_uuid: "8db96391b7384bbf983337c330332568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c98955033894142b2ce660be9f2417d" is_pre_election: true
I20260812 07:05:44.136865 5594 raft_consensus.cc:2454] T 9bc21f98b1da4cb9957e96f430e22171 P 8c98955033894142b2ce660be9f2417d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8db96391b7384bbf983337c330332568 in term 0.
I20260812 07:05:44.137290 5317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bc21f98b1da4cb9957e96f430e22171" candidate_uuid: "8db96391b7384bbf983337c330332568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6808c4ba75bb4745bb6bafada929a5ff" is_pre_election: true
I20260812 07:05:44.137580 5717 leader_election.cc:302] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c98955033894142b2ce660be9f2417d, 8db96391b7384bbf983337c330332568; no voters:
I20260812 07:05:44.137841 5317 raft_consensus.cc:2454] T 9bc21f98b1da4cb9957e96f430e22171 P 6808c4ba75bb4745bb6bafada929a5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8db96391b7384bbf983337c330332568 in term 0.
I20260812 07:05:44.137972 5970 raft_consensus.cc:2781] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:05:44.138123 5970 raft_consensus.cc:482] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:05:44.138211 5970 raft_consensus.cc:3037] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:05:44.140655 5970 raft_consensus.cc:504] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8db96391b7384bbf983337c330332568" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33705 } } peers { permanent_uuid: "6808c4ba75bb4745bb6bafada929a5ff" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43141 } } peers { permanent_uuid: "8c98955033894142b2ce660be9f2417d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 45927 } }
I20260812 07:05:44.141287 5970 leader_election.cc:288] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568 [CANDIDATE]: Term 1 election: Requested vote from peers 6808c4ba75bb4745bb6bafada929a5ff (127.11.192.193:43141), 8c98955033894142b2ce660be9f2417d (127.11.192.194:45927)
I20260812 07:05:44.141645 5317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bc21f98b1da4cb9957e96f430e22171" candidate_uuid: "8db96391b7384bbf983337c330332568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6808c4ba75bb4745bb6bafada929a5ff"
I20260812 07:05:44.141824 5317 raft_consensus.cc:3037] T 9bc21f98b1da4cb9957e96f430e22171 P 6808c4ba75bb4745bb6bafada929a5ff [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:05:44.141861 5594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bc21f98b1da4cb9957e96f430e22171" candidate_uuid: "8db96391b7384bbf983337c330332568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c98955033894142b2ce660be9f2417d"
I20260812 07:05:44.142057 5594 raft_consensus.cc:3037] T 9bc21f98b1da4cb9957e96f430e22171 P 8c98955033894142b2ce660be9f2417d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:05:44.144105 5317 raft_consensus.cc:2454] T 9bc21f98b1da4cb9957e96f430e22171 P 6808c4ba75bb4745bb6bafada929a5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8db96391b7384bbf983337c330332568 in term 1.
I20260812 07:05:44.144223 5594 raft_consensus.cc:2454] T 9bc21f98b1da4cb9957e96f430e22171 P 8c98955033894142b2ce660be9f2417d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8db96391b7384bbf983337c330332568 in term 1.
W20260812 07:05:44.144451 5413 tablet.cc:2369] T 9bc21f98b1da4cb9957e96f430e22171 P 6808c4ba75bb4745bb6bafada929a5ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:05:44.144575 5716 leader_election.cc:302] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6808c4ba75bb4745bb6bafada929a5ff, 8db96391b7384bbf983337c330332568; no voters:
I20260812 07:05:44.144948 5970 raft_consensus.cc:2781] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:05:44.145351 5970 raft_consensus.cc:686] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568 [term 1 LEADER]: Becoming Leader. State: Replica: 8db96391b7384bbf983337c330332568, State: Running, Role: LEADER
I20260812 07:05:44.145926 5970 consensus_queue.cc:237] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8db96391b7384bbf983337c330332568" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33705 } } peers { permanent_uuid: "6808c4ba75bb4745bb6bafada929a5ff" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43141 } } peers { permanent_uuid: "8c98955033894142b2ce660be9f2417d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 45927 } }
I20260812 07:05:44.150279 5037 catalog_manager.cc:5495] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8db96391b7384bbf983337c330332568 (127.11.192.195). New cstate: current_term: 1 leader_uuid: "8db96391b7384bbf983337c330332568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8db96391b7384bbf983337c330332568" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6808c4ba75bb4745bb6bafada929a5ff" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c98955033894142b2ce660be9f2417d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 45927 } health_report { overall_health: UNKNOWN } } }
W20260812 07:05:44.153839 5927 tablet.cc:2369] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:05:44.164389 5675 tablet.cc:2369] T 9bc21f98b1da4cb9957e96f430e22171 P 8c98955033894142b2ce660be9f2417d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:05:44.188936 5946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:44.184042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55304 (local address 127.11.192.195:33705)
0812 07:05:44.184229 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:44.184234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:44.184319 (+ 85us) server_negotiation.cc:407] Connection header received
0812 07:05:44.184503 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:44.184506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:44.184581 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:44.184694 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:05:44.184980 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:44.185778 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:44.187151 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:44.187539 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:44.187627 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:44.188030 (+ 403us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:05:44.188516 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:05:44.188618 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:44.188694 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 07:05:44.188755 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:05:44.196619 5317 raft_consensus.cc:1261] T 9bc21f98b1da4cb9957e96f430e22171 P 6808c4ba75bb4745bb6bafada929a5ff [term 1 FOLLOWER]: Refusing update from remote peer 8db96391b7384bbf983337c330332568: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:05:44.196615 5594 raft_consensus.cc:1261] T 9bc21f98b1da4cb9957e96f430e22171 P 8c98955033894142b2ce660be9f2417d [term 1 FOLLOWER]: Refusing update from remote peer 8db96391b7384bbf983337c330332568: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:05:44.197638 5976 consensus_queue.cc:1035] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c98955033894142b2ce660be9f2417d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 45927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:05:44.197960 5970 consensus_queue.cc:1035] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6808c4ba75bb4745bb6bafada929a5ff" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:05:44.207162 5980 mvcc.cc:205] Tried to move back new op lower bound from 7317579137819439104 to 7317579137625587712. Current Snapshot: MvccSnapshot[applied={T|T < 7317579137819439104}]
I20260812 07:05:44.207773 5982 mvcc.cc:205] Tried to move back new op lower bound from 7317579137819439104 to 7317579137625587712. Current Snapshot: MvccSnapshot[applied={T|T < 7317579137819439104}]
I20260812 07:05:44.222764 5034 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:05:44.223029 5034 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:05:44.226625 5034 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57: Sending DeleteTablet for 3 replicas of tablet 9bc21f98b1da4cb9957e96f430e22171
I20260812 07:05:44.227761 5271 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bc21f98b1da4cb9957e96f430e22171 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:05:44 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:50348
I20260812 07:05:44.227758 5551 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bc21f98b1da4cb9957e96f430e22171 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:05:44 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:41760
I20260812 07:05:44.227991 5796 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bc21f98b1da4cb9957e96f430e22171 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:05:44 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:55298
I20260812 07:05:44.228997 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 5153
I20260812 07:05:44.243427 6003 tablet_replica.cc:330] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568: stopping tablet replica
I20260812 07:05:44.243964 6003 raft_consensus.cc:2229] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:05:44.244799 6003 raft_consensus.cc:2258] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:05:44.246289 6001 tablet_replica.cc:330] T 9bc21f98b1da4cb9957e96f430e22171 P 6808c4ba75bb4745bb6bafada929a5ff: stopping tablet replica
I20260812 07:05:44.246740 6001 raft_consensus.cc:2229] T 9bc21f98b1da4cb9957e96f430e22171 P 6808c4ba75bb4745bb6bafada929a5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:05:44.247234 6001 raft_consensus.cc:2258] T 9bc21f98b1da4cb9957e96f430e22171 P 6808c4ba75bb4745bb6bafada929a5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:05:44.247960 6000 tablet_replica.cc:330] T 9bc21f98b1da4cb9957e96f430e22171 P 8c98955033894142b2ce660be9f2417d: stopping tablet replica
I20260812 07:05:44.248363 6000 raft_consensus.cc:2229] T 9bc21f98b1da4cb9957e96f430e22171 P 8c98955033894142b2ce660be9f2417d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:05:44.248799 6000 raft_consensus.cc:2258] T 9bc21f98b1da4cb9957e96f430e22171 P 8c98955033894142b2ce660be9f2417d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:05:44.249796 6003 ts_tablet_manager.cc:1905] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:05:44.250873 6001 ts_tablet_manager.cc:1905] T 9bc21f98b1da4cb9957e96f430e22171 P 6808c4ba75bb4745bb6bafada929a5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 07:05:44 dist-test-slave-lgr5 krb5kdc[4960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.193@KRBTEST.COM
I20260812 07:05:44.257375 6003 ts_tablet_manager.cc:1918] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:05:44.257613 6003 log.cc:1196] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/9bc21f98b1da4cb9957e96f430e22171
I20260812 07:05:44.258219 6003 ts_tablet_manager.cc:1939] T 9bc21f98b1da4cb9957e96f430e22171 P 8db96391b7384bbf983337c330332568: Deleting consensus metadata
I20260812 07:05:44.259631 5010 catalog_manager.cc:4841] TS 8db96391b7384bbf983337c330332568 (127.11.192.195:33705): tablet 9bc21f98b1da4cb9957e96f430e22171 (table test-table [id=50228a0c512244a9be85ed30f67e39a0]) successfully deleted
I20260812 07:05:44.260614 6000 ts_tablet_manager.cc:1905] T 9bc21f98b1da4cb9957e96f430e22171 P 8c98955033894142b2ce660be9f2417d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:05:44.266139 6001 ts_tablet_manager.cc:1918] T 9bc21f98b1da4cb9957e96f430e22171 P 6808c4ba75bb4745bb6bafada929a5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:05:44.267323 6000 ts_tablet_manager.cc:1918] T 9bc21f98b1da4cb9957e96f430e22171 P 8c98955033894142b2ce660be9f2417d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:05:44.267565 6000 log.cc:1196] T 9bc21f98b1da4cb9957e96f430e22171 P 8c98955033894142b2ce660be9f2417d: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/9bc21f98b1da4cb9957e96f430e22171
I20260812 07:05:44.268187 6000 ts_tablet_manager.cc:1939] T 9bc21f98b1da4cb9957e96f430e22171 P 8c98955033894142b2ce660be9f2417d: Deleting consensus metadata
I20260812 07:05:44.268931 5951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:44.230155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50350 (local address 127.11.192.193:43141)
0812 07:05:44.230329 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:44.230335 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:44.245693 (+ 15358us) server_negotiation.cc:407] Connection header received
0812 07:05:44.246219 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:44.246225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:44.246322 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:44.246426 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:44.248597 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:44.249739 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:44.251668 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:44.252408 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:44.252517 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:44.258091 (+ 5574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:44.258105 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:44.258122 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:44.258162 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:44.263543 (+ 5381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:44.266376 (+ 2833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:44.266385 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:44.266389 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:44.266543 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:44.267013 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:44.267019 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:44.267021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:44.267452 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:44.267755 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:44.268365 (+ 610us) server_negotiation.cc:299] Negotiation successful
0812 07:05:44.268583 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:05:44.270081 5017 catalog_manager.cc:4841] TS 8c98955033894142b2ce660be9f2417d (127.11.192.194:45927): tablet 9bc21f98b1da4cb9957e96f430e22171 (table test-table [id=50228a0c512244a9be85ed30f67e39a0]) successfully deleted
I20260812 07:05:44.270292 6001 log.cc:1196] T 9bc21f98b1da4cb9957e96f430e22171 P 6808c4ba75bb4745bb6bafada929a5ff: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/wal/wals/9bc21f98b1da4cb9957e96f430e22171
I20260812 07:05:44.270418 5395 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:05:44.357015 6001 ts_tablet_manager.cc:1939] T 9bc21f98b1da4cb9957e96f430e22171 P 6808c4ba75bb4745bb6bafada929a5ff: Deleting consensus metadata
I20260812 07:05:44.357739 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 5153
I20260812 07:05:44.381215 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 5422
W20260812 07:05:44.381177 5014 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.192.193:43141 (error 108)
I20260812 07:05:44.383929 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:44.383378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:43141 (local address 127.0.0.1:50362)
0812 07:05:44.383598 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:44.383724 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:43141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260812 07:05:44.384208 5014 catalog_manager.cc:4584] TS 6808c4ba75bb4745bb6bafada929a5ff (127.11.192.193:43141): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9bc21f98b1da4cb9957e96f430e22171: Network error: Client connection negotiation failed: client connection to 127.11.192.193:43141: connect: Connection refused (error 111)
Aug 12 07:05:44 dist-test-slave-lgr5 krb5kdc[4960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.194@KRBTEST.COM
I20260812 07:05:44.398388 5953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:44.382494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41764 (local address 127.11.192.194:45927)
0812 07:05:44.382681 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:44.382687 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:44.382707 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:05:44.382776 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:44.382782 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:44.382887 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:44.382999 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:44.384545 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:44.385394 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:44.386761 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:44.387148 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:44.387258 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:44.391968 (+ 4710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:44.391983 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:44.391998 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:44.392034 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:44.394763 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:44.395689 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:44.395697 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:44.395700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:44.395836 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:44.396247 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:44.396251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:44.396253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:44.396774 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:44.397076 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:44.397612 (+ 536us) server_negotiation.cc:299] Negotiation successful
0812 07:05:44.397960 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 07:05:44.399199 5666 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:05:44.438221 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:44.437691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:43141 (local address 127.0.0.1:50364)
0812 07:05:44.437941 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:44.438050 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:43141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260812 07:05:44.439530 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:44.439034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:43141 (local address 127.0.0.1:50366)
0812 07:05:44.439238 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:44.439346 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:43141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 07:05:44.487300 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 5422
I20260812 07:05:44.500242 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:44.499618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:43141 (local address 127.0.0.1:50382)
0812 07:05:44.499903 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:44.500035 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:43141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 07:05:44.501456 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:44.501045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:43141 (local address 127.0.0.1:50396)
0812 07:05:44.501216 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:44.501306 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:43141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 07:05:44.514153 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 5682
Aug 12 07:05:44 dist-test-slave-lgr5 krb5kdc[4960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.195@KRBTEST.COM
I20260812 07:05:44.529817 5946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:44.514951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55310 (local address 127.11.192.195:33705)
0812 07:05:44.515165 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:44.515174 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:44.515195 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:05:44.515291 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:44.515296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:44.515368 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:44.515452 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:44.516797 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:44.517600 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:44.518855 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:44.519291 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:44.519409 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:44.523988 (+ 4579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:44.524004 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:44.524021 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:44.524058 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:44.526951 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:44.527628 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:44.527635 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:44.527637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:44.527813 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:44.528204 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:44.528209 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:44.528213 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:44.528592 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:44.528917 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:44.529248 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 07:05:44.529461 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 07:05:44.530541 5914 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:05:44.581964 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:44.581528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:43141 (local address 127.0.0.1:50404)
0812 07:05:44.581735 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:44.581820 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:43141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 07:05:44.583077 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:44.582611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:43141 (local address 127.0.0.1:50408)
0812 07:05:44.582855 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:44.582939 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:43141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
I20260812 07:05:44.649917 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 5682
I20260812 07:05:44.675891 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 4982
I20260812 07:05:44.677062 5099 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:05:44.785940 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:44.785158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:43141 (local address 127.0.0.1:50412)
0812 07:05:44.785482 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:44.785607 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:43141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 07:05:44.786365 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 4982
I20260812 07:05:44.810210 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:33305
--webserver_interface=127.11.192.254
--webserver_port=46083
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:33305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:05:45.131858 6019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:45.132170 6019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:45.132311 6019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:45.139304 6019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:05:45.139396 6019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:45.139469 6019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:45.139547 6019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:05:45.139604 6019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:05:45.148936 6019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:33305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:33305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=46083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:45.150113 6019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:45.151362 6019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:05:45.174501 6025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:45.174531 6028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:45.174525 6026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:45.175122 6019 server_base.cc:1034] running on GCE node
I20260812 07:05:45.176012 6019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:45.177512 6019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:45.178719 6019 hybrid_clock.cc:648] HybridClock initialized: now 1786518345178668 us; error 35 us; skew 500 ppm
I20260812 07:05:45.179101 6019 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:45 dist-test-slave-lgr5 krb5kdc[4960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518345, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:45.182395 6019 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:05:45.183943 6019 webserver.cc:460] Webserver started at http://127.11.192.254:46083/ using document root <none> and password file <none>
I20260812 07:05:45.184707 6019 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:45.184846 6019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:05:45.190858 6019 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:05:45.193715 6037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:45.194939 6019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260812 07:05:45.195123 6019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "6ac28bd0e3db4a459678953e93aa9b57"
format_stamp: "Formatted at 2026-08-12 07:05:42 on dist-test-slave-lgr5"
server_key: "bfff20c1ef62407c83002a76b6d33ca0"
server_key_iv: "1994d35f02b725364d8cd1fd417cae8c"
server_key_version: "encryptionkey@0"
I20260812 07:05:45.196159 6019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:45.206845 6019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:45.210495 6019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:45.210744 6019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:45.228214 6019 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:33305
I20260812 07:05:45.228233 6128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:33305 every 8 connection(s)
I20260812 07:05:45.229638 6019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:05:45.232488 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 6019
I20260812 07:05:45.233438 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:43141
--local_ip_for_outbound_sockets=127.11.192.193
--tserver_master_addrs=127.11.192.254:33305
--webserver_port=43697
--webserver_interface=127.11.192.193
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:05:45.236940 6129 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:05:45.241145 6129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57: Bootstrap starting.
I20260812 07:05:45.265686 6129 log.cc:824] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57: Log is configured to *not* fsync() on all Append() calls
I20260812 07:05:45.279047 6129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:05:45.279649 6129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57: Bootstrap complete.
I20260812 07:05:45.288609 6129 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac28bd0e3db4a459678953e93aa9b57" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33305 } }
I20260812 07:05:45.289480 6129 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ac28bd0e3db4a459678953e93aa9b57, State: Initialized, Role: FOLLOWER
I20260812 07:05:45.290126 6129 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac28bd0e3db4a459678953e93aa9b57" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33305 } }
I20260812 07:05:45.290346 6129 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:05:45.290462 6129 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:05:45.290640 6129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 1 FOLLOWER]: Advancing to term 2
I20260812 07:05:45.294384 6129 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac28bd0e3db4a459678953e93aa9b57" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33305 } }
I20260812 07:05:45.294898 6129 leader_election.cc:302] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ac28bd0e3db4a459678953e93aa9b57; no voters:
I20260812 07:05:45.295511 6129 leader_election.cc:288] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 07:05:45.295738 6134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 07:05:45.296873 6134 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 2 LEADER]: Becoming Leader. State: Replica: 6ac28bd0e3db4a459678953e93aa9b57, State: Running, Role: LEADER
I20260812 07:05:45.297570 6129 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:05:45.297658 6134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac28bd0e3db4a459678953e93aa9b57" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33305 } }
I20260812 07:05:45.300519 6138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ac28bd0e3db4a459678953e93aa9b57. Latest consensus state: current_term: 2 leader_uuid: "6ac28bd0e3db4a459678953e93aa9b57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac28bd0e3db4a459678953e93aa9b57" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33305 } } }
I20260812 07:05:45.300979 6138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [sys.catalog]: This master's current role is: LEADER
I20260812 07:05:45.301594 6136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6ac28bd0e3db4a459678953e93aa9b57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac28bd0e3db4a459678953e93aa9b57" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33305 } } }
I20260812 07:05:45.301911 6136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [sys.catalog]: This master's current role is: LEADER
I20260812 07:05:45.323324 6147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:05:45.333427 6147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:05:45.334967 6147 catalog_manager.cc:1259] Loaded cluster ID: 9bd82972dc7b4fe39bbd376884eb3d01
I20260812 07:05:45.335160 6147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:05:45.338738 6147 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 07:05:45.355691 6147 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 07:05:45.359393 6170 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 07:05:45.363808 6147 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:05:45.365150 6147 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57: Loaded TSK: 0
I20260812 07:05:45.366276 6147 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 07:05:45.638758 6131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:45.639007 6131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:45.639127 6131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:45.646255 6131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:45.646355 6131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:45.646502 6131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:05:45.655959 6131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:43141
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=43697
--tserver_master_addrs=127.11.192.254:33305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:45.657157 6131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:45.658396 6131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:05:45.672560 6181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:45.672736 6179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:45.673287 6183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:45.673750 6131 server_base.cc:1034] running on GCE node
I20260812 07:05:45.674319 6131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:45.675236 6131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:45.676432 6131 hybrid_clock.cc:648] HybridClock initialized: now 1786518345676376 us; error 53 us; skew 500 ppm
I20260812 07:05:45.676719 6131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:45 dist-test-slave-lgr5 krb5kdc[4960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518345, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:45.679710 6131 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:05:45.681118 6131 webserver.cc:460] Webserver started at http://127.11.192.193:43697/ using document root <none> and password file <none>
I20260812 07:05:45.681722 6131 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:45.681847 6131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:05:45.687129 6131 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260812 07:05:45.689946 6193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:45.691341 6131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:05:45.691493 6131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "6808c4ba75bb4745bb6bafada929a5ff"
format_stamp: "Formatted at 2026-08-12 07:05:42 on dist-test-slave-lgr5"
server_key: "e16011c21a5c65ac0ad160fff8b99161"
server_key_iv: "f4ddf39f064812a829e49f7401e36b35"
server_key_version: "encryptionkey@0"
I20260812 07:05:45.692488 6131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:45.716250 6131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:45.720260 6131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:45.720500 6131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:45.721442 6131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:05:45.722790 6131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:05:45.722896 6131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:45.723006 6131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:05:45.723076 6131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:45.757323 6131 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:43141
I20260812 07:05:45.757386 6407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:43141 every 8 connection(s)
I20260812 07:05:45.758769 6131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:05:45.766480 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 6131
I20260812 07:05:45.767215 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:45927
--local_ip_for_outbound_sockets=127.11.192.194
--tserver_master_addrs=127.11.192.254:33305
--webserver_port=36067
--webserver_interface=127.11.192.194
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:05:45 dist-test-slave-lgr5 krb5kdc[4960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518345, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:05:45.785208 6410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:45.762002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:50271 (local address 127.11.192.254:33305)
0812 07:05:45.762771 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:45.762788 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:45.763317 (+ 529us) server_negotiation.cc:407] Connection header received
0812 07:05:45.765560 (+ 2243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:45.765579 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:45.766025 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:45.766437 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:45.768431 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:45.770409 (+ 1978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:45.772477 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:45.773527 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:45.773743 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:45.777987 (+ 4244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:45.777998 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:45.778015 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:45.778057 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:45.780941 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:45.781668 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:45.781674 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:45.781679 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:45.781841 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:45.782266 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:45.782269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:45.782271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:45.782718 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:45.782893 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:45.783640 (+ 747us) server_negotiation.cc:299] Negotiation successful
0812 07:05:45.784014 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":342,"threads_started":1}
I20260812 07:05:45.785210 6412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:45.762130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.193:50271)
0812 07:05:45.763118 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:45.763162 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:45.764949 (+ 1787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:45.766634 (+ 1685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:45.766645 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:45.767394 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:05:45.768136 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:45.768164 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:45.770596 (+ 2432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:45.770601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:45.772308 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:45.772324 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:45.773777 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:45.773783 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:45.774045 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:45.774801 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:05:45.774829 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:05:45.777662 (+ 2833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:05:45.781086 (+ 3424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:45.781095 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:45.781108 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:45.781527 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:45.781956 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:45.781961 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:45.781965 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:45.782147 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:45.782858 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:05:45.782866 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:05:45.783169 (+ 303us) client_negotiation.cc:769] Sending connection context
0812 07:05:45.783389 (+ 220us) client_negotiation.cc:240] Negotiation successful
0812 07:05:45.783903 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":487,"threads_started":1}
I20260812 07:05:45.787793 6408 heartbeater.cc:347] Connected to a master server at 127.11.192.254:33305
I20260812 07:05:45.788067 6408 heartbeater.cc:464] Registering TS with master...
I20260812 07:05:45.788759 6408 heartbeater.cc:511] Master 127.11.192.254:33305 requested a full tablet report, sending...
I20260812 07:05:45.790809 6068 ts_manager.cc:194] Registered new tserver with Master: 6808c4ba75bb4745bb6bafada929a5ff (127.11.192.193:43141)
I20260812 07:05:45.792279 6068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:50271
W20260812 07:05:46.097352 6413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:46.097645 6413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:46.097784 6413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:46.104874 6413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:46.104986 6413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:46.105150 6413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:05:46.114207 6413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:45927
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=36067
--tserver_master_addrs=127.11.192.254:33305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:46.115445 6413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:46.116731 6413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:05:46.129074 6419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:46.129145 6420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:46.129166 6422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:46.129966 6413 server_base.cc:1034] running on GCE node
I20260812 07:05:46.130494 6413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:46.131312 6413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:46.132508 6413 hybrid_clock.cc:648] HybridClock initialized: now 1786518346132449 us; error 48 us; skew 500 ppm
I20260812 07:05:46.132789 6413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:46 dist-test-slave-lgr5 krb5kdc[4960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518346, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:46.135651 6413 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:05:46.136934 6413 webserver.cc:460] Webserver started at http://127.11.192.194:36067/ using document root <none> and password file <none>
I20260812 07:05:46.137514 6413 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:46.137622 6413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:05:46.142678 6413 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 07:05:46.145171 6433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:46.146377 6413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:05:46.146522 6413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "8c98955033894142b2ce660be9f2417d"
format_stamp: "Formatted at 2026-08-12 07:05:43 on dist-test-slave-lgr5"
server_key: "5a0750f971e6f0e23c4fdf37a6acabfb"
server_key_iv: "6e0e64a84750a0738841d3ff384ab084"
server_key_version: "encryptionkey@0"
I20260812 07:05:46.147423 6413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:46.168967 6413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:46.172526 6413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:46.172765 6413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:46.173657 6413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:05:46.174898 6413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:05:46.175006 6413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:46.175125 6413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:05:46.175199 6413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:46.204950 6413 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:45927
I20260812 07:05:46.205025 6661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:45927 every 8 connection(s)
I20260812 07:05:46.206349 6413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:05:46.216482 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 6413
I20260812 07:05:46.217077 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:33705
--local_ip_for_outbound_sockets=127.11.192.195
--tserver_master_addrs=127.11.192.254:33305
--webserver_port=46317
--webserver_interface=127.11.192.195
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:05:46 dist-test-slave-lgr5 krb5kdc[4960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518346, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:05:46.224943 6410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:46.208658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:51487 (local address 127.11.192.254:33305)
0812 07:05:46.208824 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:46.208828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:46.209929 (+ 1101us) server_negotiation.cc:407] Connection header received
0812 07:05:46.211720 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:46.211727 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:46.211808 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:46.211896 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:46.213374 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.214119 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.215647 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.216066 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.216219 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:46.219638 (+ 3419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:46.219643 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:46.219646 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:46.219679 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:46.222037 (+ 2358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:46.222708 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:46.222713 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:46.222717 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:46.222842 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:46.223253 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:46.223257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:46.223259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:46.223525 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:46.223658 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:46.224337 (+ 679us) server_negotiation.cc:299] Negotiation successful
0812 07:05:46.224567 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:05:46.225843 6670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:46.209104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.194:51487)
0812 07:05:46.209787 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:46.209823 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:46.211444 (+ 1621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:46.212175 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:46.212183 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:46.212668 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:05:46.213196 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:46.213214 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.214284 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.214288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:46.215495 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:46.215508 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.216323 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.216330 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:46.216576 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:46.217207 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:05:46.217242 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:05:46.219427 (+ 2185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:05:46.222185 (+ 2758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:46.222191 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:46.222201 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:46.222591 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:46.222953 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:46.222957 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:46.222960 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:46.223140 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:46.223654 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:05:46.223661 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:05:46.223951 (+ 290us) client_negotiation.cc:769] Sending connection context
0812 07:05:46.224222 (+ 271us) client_negotiation.cc:240] Negotiation successful
0812 07:05:46.224711 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":266,"threads_started":1}
I20260812 07:05:46.227226 6666 heartbeater.cc:347] Connected to a master server at 127.11.192.254:33305
I20260812 07:05:46.227500 6666 heartbeater.cc:464] Registering TS with master...
I20260812 07:05:46.228327 6666 heartbeater.cc:511] Master 127.11.192.254:33305 requested a full tablet report, sending...
I20260812 07:05:46.229964 6068 ts_manager.cc:194] Registered new tserver with Master: 8c98955033894142b2ce660be9f2417d (127.11.192.194:45927)
I20260812 07:05:46.231042 6068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:51487
W20260812 07:05:46.549180 6671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:46.549468 6671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:46.549618 6671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:46.556851 6671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:46.556986 6671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:46.557156 6671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:05:46.566712 6671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:33705
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=46317
--tserver_master_addrs=127.11.192.254:33305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:46.567909 6671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:46.569235 6671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:05:46.582019 6679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:46.581969 6680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:46.581939 6684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:46.583062 6671 server_base.cc:1034] running on GCE node
I20260812 07:05:46.583616 6671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:46.584502 6671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:46.585687 6671 hybrid_clock.cc:648] HybridClock initialized: now 1786518346585642 us; error 40 us; skew 500 ppm
I20260812 07:05:46.585949 6671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:46 dist-test-slave-lgr5 krb5kdc[4960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518346, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:46.596163 6671 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:05:46.597520 6671 webserver.cc:460] Webserver started at http://127.11.192.195:46317/ using document root <none> and password file <none>
I20260812 07:05:46.598188 6671 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:46.598305 6671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:05:46.603458 6671 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:05:46.606019 6693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:46.607239 6671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:05:46.607366 6671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "8db96391b7384bbf983337c330332568"
format_stamp: "Formatted at 2026-08-12 07:05:43 on dist-test-slave-lgr5"
server_key: "1ec9dfbf108e524251d730a2bd9ca28b"
server_key_iv: "b4bedc7c48b21f015a77a8c4fad05a27"
server_key_version: "encryptionkey@0"
I20260812 07:05:46.608304 6671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:46.626695 6671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:46.630527 6671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:46.630728 6671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:46.631567 6671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:05:46.632799 6671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:05:46.632890 6671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:46.632968 6671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:05:46.633051 6671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:46.662446 6671 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:33705
I20260812 07:05:46.662503 6880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:33705 every 8 connection(s)
I20260812 07:05:46.663826 6671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:05:46.664377 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 6671
Aug 12 07:05:46 dist-test-slave-lgr5 krb5kdc[4960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518346, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:05:46.679478 6410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:46.665464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36342 (local address 127.11.192.254:33305)
0812 07:05:46.665664 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:46.665670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:46.666179 (+ 509us) server_negotiation.cc:407] Connection header received
0812 07:05:46.666342 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:46.666349 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:46.666423 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:46.666514 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:46.668129 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.668882 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.670172 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.670605 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.670712 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:46.673267 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:46.673272 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:46.673275 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:46.673306 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:46.676003 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:46.676768 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:46.676772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:46.676775 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:46.676893 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:46.677959 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:46.677965 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:46.677969 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:46.678285 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:46.678554 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:46.678884 (+ 330us) server_negotiation.cc:299] Negotiation successful
0812 07:05:46.679044 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 07:05:46.683439 6889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:46.666437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:58413 (local address 127.11.192.254:33305)
0812 07:05:46.666948 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:46.666953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:46.668129 (+ 1176us) server_negotiation.cc:407] Connection header received
0812 07:05:46.669838 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:46.669842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:46.669911 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:46.670005 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:46.671429 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.672145 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.673641 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.674250 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.674360 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:46.677710 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:46.677714 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:46.677717 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:46.677745 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:46.680836 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:46.681515 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:46.681520 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:46.681523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:46.681674 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:46.682138 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:46.682142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:46.682144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:46.682387 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:46.682498 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:46.683039 (+ 541us) server_negotiation.cc:299] Negotiation successful
0812 07:05:46.683174 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":279,"threads_started":1}
I20260812 07:05:46.684402 6890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:46.667110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.195:58413)
0812 07:05:46.667892 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:46.667945 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:46.669593 (+ 1648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:46.670272 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:46.670279 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:46.670746 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:05:46.671256 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:46.671272 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.672308 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.672312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:46.673494 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:46.673506 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.674494 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.674503 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:46.674706 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:46.675276 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:05:46.675302 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:05:46.677535 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:05:46.680968 (+ 3433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:46.680977 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:46.681006 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:46.681394 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:46.681778 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:05:46.681783 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:05:46.681787 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:05:46.682011 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:05:46.682494 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:05:46.682499 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:05:46.682772 (+ 273us) client_negotiation.cc:769] Sending connection context
0812 07:05:46.682985 (+ 213us) client_negotiation.cc:240] Negotiation successful
0812 07:05:46.683317 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":317,"threads_started":1}
I20260812 07:05:46.685606 6882 heartbeater.cc:347] Connected to a master server at 127.11.192.254:33305
I20260812 07:05:46.685873 6882 heartbeater.cc:464] Registering TS with master...
I20260812 07:05:46.686558 6882 heartbeater.cc:511] Master 127.11.192.254:33305 requested a full tablet report, sending...
I20260812 07:05:46.687830 6068 ts_manager.cc:194] Registered new tserver with Master: 8db96391b7384bbf983337c330332568 (127.11.192.195:33705)
I20260812 07:05:46.688633 6068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:58413
I20260812 07:05:46.693984 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:05:46.700407 6889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:46.694827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36358 (local address 127.11.192.254:33305)
0812 07:05:46.694971 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:46.694977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:46.695511 (+ 534us) server_negotiation.cc:407] Connection header received
0812 07:05:46.695703 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:46.695707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:46.695839 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:46.695951 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:05:46.696335 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.697048 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.698439 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.698828 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.698915 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:46.699379 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:05:46.699852 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:05:46.699948 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:46.700038 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 07:05:46.700147 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:05:46.701179 6068 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36358:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:05:46.708173 6068 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:05:46.734522 6901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:46.719149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55318 (local address 127.11.192.195:33705)
0812 07:05:46.719856 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:46.719863 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:46.724841 (+ 4978us) server_negotiation.cc:407] Connection header received
0812 07:05:46.725264 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:46.725273 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:46.725533 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:46.725687 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:05:46.727095 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.728861 (+ 1766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.731802 (+ 2941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.733336 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.733522 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:46.733705 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:46.734045 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 07:05:46.734199 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":401,"threads_started":1}
I20260812 07:05:46.736276 6900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:46.718909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:33705 (local address 127.0.0.1:55318)
0812 07:05:46.724695 (+ 5786us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:46.724735 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:46.725025 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:46.726072 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:46.726080 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:46.726113 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:05:46.726858 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:46.726872 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.729082 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.729087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:46.731585 (+ 2498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:46.731602 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.733592 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.733599 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:46.733801 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:46.733852 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 07:05:46.733986 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 07:05:46.734184 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5533,"thread_start_us":5388,"threads_started":1}
I20260812 07:05:46.740195 6778 tablet_service.cc:1467] Processing CreateTablet for tablet 88f81f66fc6c4388956a559829bf1489 (DEFAULT_TABLE table=test-table [id=5765ed37a02f49d7890e697cb585f983]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:05:46.741021 6905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:46.719929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50432 (local address 127.11.192.193:43141)
0812 07:05:46.721025 (+ 1096us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:46.721033 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:46.732067 (+ 11034us) server_negotiation.cc:407] Connection header received
0812 07:05:46.733255 (+ 1188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:46.733263 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:46.733490 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:46.733656 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:05:46.734420 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.735967 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.738402 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.739764 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.739921 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:46.740114 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:46.740554 (+ 440us) server_negotiation.cc:299] Negotiation successful
0812 07:05:46.740722 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":940,"thread_start_us":818,"threads_started":1}
I20260812 07:05:46.741021 6904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:46.719264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:43141 (local address 127.0.0.1:50432)
0812 07:05:46.731937 (+ 12673us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:46.731956 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:46.733040 (+ 1084us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:46.733823 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:46.733827 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:46.733851 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:05:46.734225 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:46.734239 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.736129 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.736134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:46.738249 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:46.738262 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.740121 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.740129 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:46.740359 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:46.740384 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:05:46.740465 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 07:05:46.740584 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12541,"mutex_wait_us":22,"thread_start_us":13974,"threads_started":1}
I20260812 07:05:46.741744 6778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88f81f66fc6c4388956a559829bf1489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:05:46.743144 6902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:46.719521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:45927 (local address 127.0.0.1:41778)
0812 07:05:46.733519 (+ 13998us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:46.733536 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:46.733649 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:46.734513 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:46.734518 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:46.734543 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:05:46.734929 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:46.734941 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.737794 (+ 2853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.737799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:46.740496 (+ 2697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:46.740511 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.742336 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.742344 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:46.742509 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:46.742541 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 07:05:46.742654 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 07:05:46.742773 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13879,"mutex_wait_us":28,"thread_start_us":12594,"threads_started":1}
I20260812 07:05:46.743780 6906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:46.720431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41778 (local address 127.11.192.194:45927)
0812 07:05:46.732660 (+ 12229us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:46.732668 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:46.733770 (+ 1102us) server_negotiation.cc:407] Connection header received
0812 07:05:46.733866 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:46.733872 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:46.734100 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:46.734281 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:05:46.735193 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.737641 (+ 2448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.740671 (+ 3030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.742118 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.742279 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:46.742438 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:46.743354 (+ 916us) server_negotiation.cc:299] Negotiation successful
0812 07:05:46.743508 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12074,"thread_start_us":12237,"threads_started":1}
I20260812 07:05:46.744568 6290 tablet_service.cc:1467] Processing CreateTablet for tablet 88f81f66fc6c4388956a559829bf1489 (DEFAULT_TABLE table=test-table [id=5765ed37a02f49d7890e697cb585f983]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:05:46.745843 6290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88f81f66fc6c4388956a559829bf1489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:05:46.747175 6530 tablet_service.cc:1467] Processing CreateTablet for tablet 88f81f66fc6c4388956a559829bf1489 (DEFAULT_TABLE table=test-table [id=5765ed37a02f49d7890e697cb585f983]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:05:46.748420 6530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88f81f66fc6c4388956a559829bf1489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:05:46.754721 6907 tablet_bootstrap.cc:492] T 88f81f66fc6c4388956a559829bf1489 P 8db96391b7384bbf983337c330332568: Bootstrap starting.
I20260812 07:05:46.758296 6907 tablet_bootstrap.cc:654] T 88f81f66fc6c4388956a559829bf1489 P 8db96391b7384bbf983337c330332568: Neither blocks nor log segments found. Creating new log.
I20260812 07:05:46.760483 6907 log.cc:824] T 88f81f66fc6c4388956a559829bf1489 P 8db96391b7384bbf983337c330332568: Log is configured to *not* fsync() on all Append() calls
I20260812 07:05:46.767584 6909 tablet_bootstrap.cc:492] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff: Bootstrap starting.
I20260812 07:05:46.768302 6916 tablet_bootstrap.cc:492] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d: Bootstrap starting.
I20260812 07:05:46.769976 6907 tablet_bootstrap.cc:492] T 88f81f66fc6c4388956a559829bf1489 P 8db96391b7384bbf983337c330332568: No bootstrap required, opened a new log
I20260812 07:05:46.770251 6907 ts_tablet_manager.cc:1397] T 88f81f66fc6c4388956a559829bf1489 P 8db96391b7384bbf983337c330332568: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.008s
I20260812 07:05:46.771270 6909 tablet_bootstrap.cc:654] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff: Neither blocks nor log segments found. Creating new log.
I20260812 07:05:46.771955 6916 tablet_bootstrap.cc:654] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d: Neither blocks nor log segments found. Creating new log.
I20260812 07:05:46.772248 6909 log.cc:824] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff: Log is configured to *not* fsync() on all Append() calls
I20260812 07:05:46.773177 6916 log.cc:824] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d: Log is configured to *not* fsync() on all Append() calls
I20260812 07:05:46.774936 6909 tablet_bootstrap.cc:492] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff: No bootstrap required, opened a new log
I20260812 07:05:46.775218 6909 ts_tablet_manager.cc:1397] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 07:05:46.778420 6916 tablet_bootstrap.cc:492] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d: No bootstrap required, opened a new log
I20260812 07:05:46.778707 6916 ts_tablet_manager.cc:1397] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260812 07:05:46.782871 6907 raft_consensus.cc:348] T 88f81f66fc6c4388956a559829bf1489 P 8db96391b7384bbf983337c330332568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8db96391b7384bbf983337c330332568" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33705 } } peers { permanent_uuid: "6808c4ba75bb4745bb6bafada929a5ff" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43141 } } peers { permanent_uuid: "8c98955033894142b2ce660be9f2417d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 45927 } }
I20260812 07:05:46.783407 6907 raft_consensus.cc:374] T 88f81f66fc6c4388956a559829bf1489 P 8db96391b7384bbf983337c330332568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:05:46.783551 6907 raft_consensus.cc:729] T 88f81f66fc6c4388956a559829bf1489 P 8db96391b7384bbf983337c330332568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8db96391b7384bbf983337c330332568, State: Initialized, Role: FOLLOWER
I20260812 07:05:46.784315 6907 consensus_queue.cc:260] T 88f81f66fc6c4388956a559829bf1489 P 8db96391b7384bbf983337c330332568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8db96391b7384bbf983337c330332568" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33705 } } peers { permanent_uuid: "6808c4ba75bb4745bb6bafada929a5ff" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43141 } } peers { permanent_uuid: "8c98955033894142b2ce660be9f2417d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 45927 } }
I20260812 07:05:46.785146 6909 raft_consensus.cc:348] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8db96391b7384bbf983337c330332568" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33705 } } peers { permanent_uuid: "6808c4ba75bb4745bb6bafada929a5ff" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43141 } } peers { permanent_uuid: "8c98955033894142b2ce660be9f2417d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 45927 } }
I20260812 07:05:46.785485 6909 raft_consensus.cc:374] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:05:46.785559 6909 raft_consensus.cc:729] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6808c4ba75bb4745bb6bafada929a5ff, State: Initialized, Role: FOLLOWER
I20260812 07:05:46.786065 6909 consensus_queue.cc:260] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8db96391b7384bbf983337c330332568" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33705 } } peers { permanent_uuid: "6808c4ba75bb4745bb6bafada929a5ff" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43141 } } peers { permanent_uuid: "8c98955033894142b2ce660be9f2417d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 45927 } }
I20260812 07:05:46.787400 6408 heartbeater.cc:503] Master 127.11.192.254:33305 was elected leader, sending a full tablet report...
I20260812 07:05:46.788153 6909 ts_tablet_manager.cc:1428] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260812 07:05:46.789124 6916 raft_consensus.cc:348] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8db96391b7384bbf983337c330332568" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33705 } } peers { permanent_uuid: "6808c4ba75bb4745bb6bafada929a5ff" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43141 } } peers { permanent_uuid: "8c98955033894142b2ce660be9f2417d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 45927 } }
I20260812 07:05:46.789621 6916 raft_consensus.cc:374] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:05:46.789750 6916 raft_consensus.cc:729] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c98955033894142b2ce660be9f2417d, State: Initialized, Role: FOLLOWER
I20260812 07:05:46.789794 6907 ts_tablet_manager.cc:1428] T 88f81f66fc6c4388956a559829bf1489 P 8db96391b7384bbf983337c330332568: Time spent starting tablet: real 0.019s user 0.015s sys 0.004s
I20260812 07:05:46.790360 6882 heartbeater.cc:503] Master 127.11.192.254:33305 was elected leader, sending a full tablet report...
I20260812 07:05:46.791694 6916 consensus_queue.cc:260] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8db96391b7384bbf983337c330332568" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33705 } } peers { permanent_uuid: "6808c4ba75bb4745bb6bafada929a5ff" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43141 } } peers { permanent_uuid: "8c98955033894142b2ce660be9f2417d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 45927 } }
I20260812 07:05:46.795482 6916 ts_tablet_manager.cc:1428] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d: Time spent starting tablet: real 0.017s user 0.009s sys 0.006s
I20260812 07:05:46.795971 6666 heartbeater.cc:503] Master 127.11.192.254:33305 was elected leader, sending a full tablet report...
I20260812 07:05:46.796108 6928 raft_consensus.cc:482] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:05:46.796365 6928 raft_consensus.cc:504] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8db96391b7384bbf983337c330332568" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33705 } } peers { permanent_uuid: "6808c4ba75bb4745bb6bafada929a5ff" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43141 } } peers { permanent_uuid: "8c98955033894142b2ce660be9f2417d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 45927 } }
I20260812 07:05:46.798454 6928 leader_election.cc:288] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8db96391b7384bbf983337c330332568 (127.11.192.195:33705), 6808c4ba75bb4745bb6bafada929a5ff (127.11.192.193:43141)
I20260812 07:05:46.815425 6905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:46.799381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:57723 (local address 127.11.192.193:43141)
0812 07:05:46.799556 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:46.799562 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:46.809048 (+ 9486us) server_negotiation.cc:407] Connection header received
0812 07:05:46.809244 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:46.809248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:46.809304 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:46.809350 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:05:46.810054 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.810856 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.813346 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.814515 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.814666 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:46.814739 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:46.815059 (+ 320us) server_negotiation.cc:299] Negotiation successful
0812 07:05:46.815183 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:05:46.815570 6938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:46.799370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:43141 (local address 127.11.192.194:57723)
0812 07:05:46.808954 (+ 9584us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:46.808968 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:46.809132 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:46.809503 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:46.809507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:46.809526 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:05:46.809891 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:46.809899 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.810993 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.810998 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:46.813170 (+ 2172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:46.813185 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.814725 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.814731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:46.814887 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:46.814911 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 07:05:46.815008 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 07:05:46.815161 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9458,"mutex_wait_us":53,"thread_start_us":9261,"threads_started":1}
I20260812 07:05:46.815667 6901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:46.799161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:45263 (local address 127.11.192.195:33705)
0812 07:05:46.799438 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:46.799445 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:46.808613 (+ 9168us) server_negotiation.cc:407] Connection header received
0812 07:05:46.808792 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:46.808797 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:46.808879 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:46.808975 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:05:46.809868 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.811058 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.813483 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.814649 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.814780 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:46.814839 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:46.815312 (+ 473us) server_negotiation.cc:299] Negotiation successful
0812 07:05:46.815432 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 07:05:46.816295 6936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:46.798803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:33705 (local address 127.11.192.194:45263)
0812 07:05:46.808511 (+ 9708us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:46.808529 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:05:46.808663 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:05:46.809175 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:05:46.809181 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:05:46.809229 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:05:46.809639 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:46.809659 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.811191 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.811197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:46.813297 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:05:46.813313 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.815006 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.815011 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:05:46.815159 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:46.815180 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 07:05:46.815258 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 07:05:46.815380 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9579,"mutex_wait_us":48,"thread_start_us":9455,"threads_started":1}
I20260812 07:05:46.816691 6330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88f81f66fc6c4388956a559829bf1489" candidate_uuid: "8c98955033894142b2ce660be9f2417d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6808c4ba75bb4745bb6bafada929a5ff" is_pre_election: true
I20260812 07:05:46.817163 6330 raft_consensus.cc:2454] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c98955033894142b2ce660be9f2417d in term 0.
I20260812 07:05:46.817821 6440 leader_election.cc:302] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6808c4ba75bb4745bb6bafada929a5ff, 8c98955033894142b2ce660be9f2417d; no voters:
I20260812 07:05:46.818216 6929 raft_consensus.cc:2781] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:05:46.818363 6929 raft_consensus.cc:482] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:05:46.818497 6929 raft_consensus.cc:3037] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:05:46.818321 6813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88f81f66fc6c4388956a559829bf1489" candidate_uuid: "8c98955033894142b2ce660be9f2417d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8db96391b7384bbf983337c330332568" is_pre_election: true
I20260812 07:05:46.818846 6813 raft_consensus.cc:2454] T 88f81f66fc6c4388956a559829bf1489 P 8db96391b7384bbf983337c330332568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c98955033894142b2ce660be9f2417d in term 0.
I20260812 07:05:46.820966 6929 raft_consensus.cc:504] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8db96391b7384bbf983337c330332568" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33705 } } peers { permanent_uuid: "6808c4ba75bb4745bb6bafada929a5ff" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43141 } } peers { permanent_uuid: "8c98955033894142b2ce660be9f2417d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 45927 } }
I20260812 07:05:46.821894 6330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88f81f66fc6c4388956a559829bf1489" candidate_uuid: "8c98955033894142b2ce660be9f2417d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6808c4ba75bb4745bb6bafada929a5ff"
I20260812 07:05:46.822052 6330 raft_consensus.cc:3037] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:05:46.823933 6813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88f81f66fc6c4388956a559829bf1489" candidate_uuid: "8c98955033894142b2ce660be9f2417d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8db96391b7384bbf983337c330332568"
I20260812 07:05:46.824159 6813 raft_consensus.cc:3037] T 88f81f66fc6c4388956a559829bf1489 P 8db96391b7384bbf983337c330332568 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:05:46.824354 6330 raft_consensus.cc:2454] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c98955033894142b2ce660be9f2417d in term 1.
I20260812 07:05:46.824638 6929 leader_election.cc:288] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d [CANDIDATE]: Term 1 election: Requested vote from peers 8db96391b7384bbf983337c330332568 (127.11.192.195:33705), 6808c4ba75bb4745bb6bafada929a5ff (127.11.192.193:43141)
I20260812 07:05:46.824815 6440 leader_election.cc:302] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6808c4ba75bb4745bb6bafada929a5ff, 8c98955033894142b2ce660be9f2417d; no voters:
I20260812 07:05:46.825059 6929 raft_consensus.cc:2781] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:05:46.825254 6929 raft_consensus.cc:686] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d [term 1 LEADER]: Becoming Leader. State: Replica: 8c98955033894142b2ce660be9f2417d, State: Running, Role: LEADER
I20260812 07:05:46.825745 6929 consensus_queue.cc:237] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8db96391b7384bbf983337c330332568" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33705 } } peers { permanent_uuid: "6808c4ba75bb4745bb6bafada929a5ff" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43141 } } peers { permanent_uuid: "8c98955033894142b2ce660be9f2417d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 45927 } }
I20260812 07:05:46.826726 6813 raft_consensus.cc:2454] T 88f81f66fc6c4388956a559829bf1489 P 8db96391b7384bbf983337c330332568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c98955033894142b2ce660be9f2417d in term 1.
I20260812 07:05:46.830750 6068 catalog_manager.cc:5495] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c98955033894142b2ce660be9f2417d (127.11.192.194). New cstate: current_term: 1 leader_uuid: "8c98955033894142b2ce660be9f2417d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8db96391b7384bbf983337c330332568" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6808c4ba75bb4745bb6bafada929a5ff" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c98955033894142b2ce660be9f2417d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 45927 } health_report { overall_health: HEALTHY } } }
I20260812 07:05:46.860872 6906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:46.856037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41790 (local address 127.11.192.194:45927)
0812 07:05:46.856234 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:46.856238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:46.856379 (+ 141us) server_negotiation.cc:407] Connection header received
0812 07:05:46.856454 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:46.856458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:46.856539 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:46.856651 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:05:46.856946 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.857653 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.858985 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.859368 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.859452 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:46.859822 (+ 370us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:05:46.860355 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:05:46.860458 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:46.860546 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 07:05:46.860609 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:05:46.868310 6813 raft_consensus.cc:1261] T 88f81f66fc6c4388956a559829bf1489 P 8db96391b7384bbf983337c330332568 [term 1 FOLLOWER]: Refusing update from remote peer 8c98955033894142b2ce660be9f2417d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:05:46.868366 6330 raft_consensus.cc:1261] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff [term 1 FOLLOWER]: Refusing update from remote peer 8c98955033894142b2ce660be9f2417d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:05:46.869172 6929 consensus_queue.cc:1035] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8db96391b7384bbf983337c330332568" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:05:46.869450 6928 consensus_queue.cc:1035] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6808c4ba75bb4745bb6bafada929a5ff" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:05:46.879357 6942 mvcc.cc:205] Tried to move back new op lower bound from 7317579148763385856 to 7317579148602281984. Current Snapshot: MvccSnapshot[applied={T|T < 7317579148763385856}]
I20260812 07:05:46.881775 6944 mvcc.cc:205] Tried to move back new op lower bound from 7317579148763385856 to 7317579148602281984. Current Snapshot: MvccSnapshot[applied={T|T < 7317579148763385856}]
I20260812 07:05:46.892889 6067 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:05:46.893193 6067 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:05:46.897205 6067 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57: Sending DeleteTablet for 3 replicas of tablet 88f81f66fc6c4388956a559829bf1489
I20260812 07:05:46.898307 6778 tablet_service.cc:1514] Processing DeleteTablet for tablet 88f81f66fc6c4388956a559829bf1489 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:05:46 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:55318
I20260812 07:05:46.898383 6290 tablet_service.cc:1514] Processing DeleteTablet for tablet 88f81f66fc6c4388956a559829bf1489 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:05:46 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:50432
I20260812 07:05:46.898739 6530 tablet_service.cc:1514] Processing DeleteTablet for tablet 88f81f66fc6c4388956a559829bf1489 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:05:46 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:41778
I20260812 07:05:46.899840 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 6131
I20260812 07:05:46.908514 6965 tablet_replica.cc:330] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d: stopping tablet replica
I20260812 07:05:46.908664 6964 tablet_replica.cc:330] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff: stopping tablet replica
I20260812 07:05:46.908942 6965 raft_consensus.cc:2229] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:05:46.909049 6964 raft_consensus.cc:2229] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:05:46.909538 6964 raft_consensus.cc:2258] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:05:46.909557 6965 raft_consensus.cc:2258] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:05:46.912150 6965 ts_tablet_manager.cc:1905] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:05:46.917623 6964 ts_tablet_manager.cc:1905] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:05:46.919737 6965 ts_tablet_manager.cc:1918] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:05:46.919904 6965 log.cc:1196] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/88f81f66fc6c4388956a559829bf1489
I20260812 07:05:46.920490 6965 ts_tablet_manager.cc:1939] T 88f81f66fc6c4388956a559829bf1489 P 8c98955033894142b2ce660be9f2417d: Deleting consensus metadata
I20260812 07:05:46.920809 6905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:46.900712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50434 (local address 127.11.192.193:43141)
0812 07:05:46.901219 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:46.901225 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:46.901242 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:05:46.901295 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:46.901300 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:46.901376 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:46.901468 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:46.903401 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.904595 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.906542 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:46.907174 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:46.907294 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:46.911371 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:46.911387 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:46.911403 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:46.911442 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:46.916671 (+ 5229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:46.917649 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:46.917658 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:46.917661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:46.917864 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:46.918327 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:46.918332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:46.918335 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:46.918817 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:46.919128 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:46.920166 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0812 07:05:46.920374 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385}
I20260812 07:05:46.922042 6399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:05:46.922248 6045 catalog_manager.cc:4841] TS 8c98955033894142b2ce660be9f2417d (127.11.192.194:45927): tablet 88f81f66fc6c4388956a559829bf1489 (table test-table [id=5765ed37a02f49d7890e697cb585f983]) successfully deleted
I20260812 07:05:46.922454 6963 tablet_replica.cc:330] T 88f81f66fc6c4388956a559829bf1489 P 8db96391b7384bbf983337c330332568: stopping tablet replica
I20260812 07:05:46.922830 6963 raft_consensus.cc:2229] T 88f81f66fc6c4388956a559829bf1489 P 8db96391b7384bbf983337c330332568 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:05:46.923324 6963 raft_consensus.cc:2258] T 88f81f66fc6c4388956a559829bf1489 P 8db96391b7384bbf983337c330332568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:05:46.925053 6963 ts_tablet_manager.cc:1905] T 88f81f66fc6c4388956a559829bf1489 P 8db96391b7384bbf983337c330332568: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:05:46.931720 6963 ts_tablet_manager.cc:1918] T 88f81f66fc6c4388956a559829bf1489 P 8db96391b7384bbf983337c330332568: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:05:46.931881 6963 log.cc:1196] T 88f81f66fc6c4388956a559829bf1489 P 8db96391b7384bbf983337c330332568: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/88f81f66fc6c4388956a559829bf1489
I20260812 07:05:46.932466 6963 ts_tablet_manager.cc:1939] T 88f81f66fc6c4388956a559829bf1489 P 8db96391b7384bbf983337c330332568: Deleting consensus metadata
I20260812 07:05:46.934172 6042 catalog_manager.cc:4841] TS 8db96391b7384bbf983337c330332568 (127.11.192.195:33705): tablet 88f81f66fc6c4388956a559829bf1489 (table test-table [id=5765ed37a02f49d7890e697cb585f983]) successfully deleted
I20260812 07:05:47.012248 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 6131
W20260812 07:05:47.036231 6044 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.192.193:43141 (error 108)
I20260812 07:05:47.036676 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 6413
I20260812 07:05:47.038656 6902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:47.038261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:43141 (local address 127.0.0.1:50436)
0812 07:05:47.038408 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:47.038519 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:43141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 07:05:47.038949 6044 catalog_manager.cc:4584] TS 6808c4ba75bb4745bb6bafada929a5ff (127.11.192.193:43141): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 88f81f66fc6c4388956a559829bf1489: Network error: Client connection negotiation failed: client connection to 127.11.192.193:43141: connect: Connection refused (error 111)
I20260812 07:05:47.052460 6906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:47.037503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41792 (local address 127.11.192.194:45927)
0812 07:05:47.037688 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:47.037694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:47.037744 (+ 50us) server_negotiation.cc:407] Connection header received
0812 07:05:47.037951 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:47.037955 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:47.038037 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:47.038143 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:47.040159 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:47.041021 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:47.042877 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:47.043309 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:47.043396 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:47.046651 (+ 3255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:47.046664 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:47.046677 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:47.046708 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:47.049592 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:47.050423 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:47.050432 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:47.050435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:47.050603 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:47.051014 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:47.051017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:47.051019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:47.051429 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:47.051692 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:47.051980 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 07:05:47.052179 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 07:05:47.053067 6654 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:05:47.092597 6902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:47.092020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:43141 (local address 127.0.0.1:50440)
0812 07:05:47.092272 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:47.092397 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:43141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260812 07:05:47.093658 6902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:47.093254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:43141 (local address 127.0.0.1:50456)
0812 07:05:47.093409 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:47.093521 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:43141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 07:05:47.154261 6902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:47.153670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:43141 (local address 127.0.0.1:50458)
0812 07:05:47.153935 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:47.154051 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:43141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260812 07:05:47.155354 6902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:47.154941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:43141 (local address 127.0.0.1:50468)
0812 07:05:47.155124 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:47.155213 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:43141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 07:05:47.164431 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 6413
I20260812 07:05:47.189710 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 6671
I20260812 07:05:47.205358 6901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:47.190378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55324 (local address 127.11.192.195:33705)
0812 07:05:47.190543 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 07:05:47.190548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:05:47.190626 (+ 78us) server_negotiation.cc:407] Connection header received
0812 07:05:47.190773 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:05:47.190777 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:05:47.190841 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:05:47.190938 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:05:47.192216 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:47.192951 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:47.194270 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:05:47.194646 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:05:47.194732 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:05:47.200425 (+ 5693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:05:47.200438 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:05:47.200454 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:05:47.200493 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:05:47.202998 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:47.203609 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:47.203616 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:47.203618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:47.203751 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:05:47.204028 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:05:47.204031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:05:47.204033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:05:47.204469 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:05:47.204704 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 07:05:47.204953 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 07:05:47.205118 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:05:47.206013 6872 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:05:47.235950 6902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:47.235456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:43141 (local address 127.0.0.1:50480)
0812 07:05:47.235668 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:47.235776 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:43141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 07:05:47.237195 6900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:47.236692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:43141 (local address 127.0.0.1:50494)
0812 07:05:47.236933 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:47.237039 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:43141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260812 07:05:47.292776 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 6671
I20260812 07:05:47.317178 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 6019
I20260812 07:05:47.318275 6119 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:05:47.430485 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 6019
I20260812 07:05:47.458985 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:33305
--webserver_interface=127.11.192.254
--webserver_port=46083
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:33305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:05:47.779811 6973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:47.780102 6973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:47.780242 6973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:47.787209 6973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:05:47.787303 6973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:47.787377 6973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:47.787453 6973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:05:47.787511 6973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:05:47.796943 6973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:33305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:33305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=46083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:47.798179 6973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:47.799431 6973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:05:47.810189 6987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:47.810235 6985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:47.810527 6982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:47.811333 6973 server_base.cc:1034] running on GCE node
I20260812 07:05:47.812037 6973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:47.813345 6973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:47.814551 6973 hybrid_clock.cc:648] HybridClock initialized: now 1786518347814501 us; error 40 us; skew 500 ppm
I20260812 07:05:47.814865 6973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:47 dist-test-slave-lgr5 krb5kdc[4960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518347, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:47.817799 6973 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:05:47.819155 6973 webserver.cc:460] Webserver started at http://127.11.192.254:46083/ using document root <none> and password file <none>
I20260812 07:05:47.819768 6973 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:47.819891 6973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:05:47.824858 6973 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260812 07:05:47.827409 6997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:47.828615 6973 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260812 07:05:47.828771 6973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "6ac28bd0e3db4a459678953e93aa9b57"
format_stamp: "Formatted at 2026-08-12 07:05:42 on dist-test-slave-lgr5"
server_key: "bfff20c1ef62407c83002a76b6d33ca0"
server_key_iv: "1994d35f02b725364d8cd1fd417cae8c"
server_key_version: "encryptionkey@0"
I20260812 07:05:47.829663 6973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:47.846819 6973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:47.850574 6973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:47.850802 6973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:47.867707 6973 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:33305
I20260812 07:05:47.867713 7086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:33305 every 8 connection(s)
I20260812 07:05:47.869117 6973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:05:47.872028 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 6973
I20260812 07:05:47.872840 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:43141
--local_ip_for_outbound_sockets=127.11.192.193
--tserver_master_addrs=127.11.192.254:33305
--webserver_port=43697
--webserver_interface=127.11.192.193
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:05:47.875607 7087 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:05:47.878726 7087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57: Bootstrap starting.
I20260812 07:05:47.898223 7087 log.cc:824] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57: Log is configured to *not* fsync() on all Append() calls
I20260812 07:05:47.913823 7087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:05:47.914289 7087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57: Bootstrap complete.
I20260812 07:05:47.922914 7087 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac28bd0e3db4a459678953e93aa9b57" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33305 } }
I20260812 07:05:47.923741 7087 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ac28bd0e3db4a459678953e93aa9b57, State: Initialized, Role: FOLLOWER
I20260812 07:05:47.924343 7087 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac28bd0e3db4a459678953e93aa9b57" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33305 } }
I20260812 07:05:47.924576 7087 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:05:47.924683 7087 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:05:47.924850 7087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 2 FOLLOWER]: Advancing to term 3
I20260812 07:05:47.928398 7087 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac28bd0e3db4a459678953e93aa9b57" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33305 } }
I20260812 07:05:47.928857 7087 leader_election.cc:302] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ac28bd0e3db4a459678953e93aa9b57; no voters:
I20260812 07:05:47.929450 7087 leader_election.cc:288] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 07:05:47.929576 7091 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 07:05:47.930552 7091 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [term 3 LEADER]: Becoming Leader. State: Replica: 6ac28bd0e3db4a459678953e93aa9b57, State: Running, Role: LEADER
I20260812 07:05:47.931094 7091 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac28bd0e3db4a459678953e93aa9b57" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33305 } }
I20260812 07:05:47.931459 7087 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:05:47.933579 7093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ac28bd0e3db4a459678953e93aa9b57. Latest consensus state: current_term: 3 leader_uuid: "6ac28bd0e3db4a459678953e93aa9b57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac28bd0e3db4a459678953e93aa9b57" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33305 } } }
I20260812 07:05:47.933589 7092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6ac28bd0e3db4a459678953e93aa9b57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac28bd0e3db4a459678953e93aa9b57" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33305 } } }
I20260812 07:05:47.933959 7093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [sys.catalog]: This master's current role is: LEADER
I20260812 07:05:47.934041 7092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ac28bd0e3db4a459678953e93aa9b57 [sys.catalog]: This master's current role is: LEADER
I20260812 07:05:47.945737 7098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:05:47.954496 7098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:05:47.956027 7098 catalog_manager.cc:1259] Loaded cluster ID: 9bd82972dc7b4fe39bbd376884eb3d01
I20260812 07:05:47.956141 7098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 07:05:47.958362 7098 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f791b8519ad google::LogMessage::Fail() at ??:0
@ 0x7f791b85488a google::LogMessage::SendToLog() at ??:0
@ 0x7f791b8514de google::LogMessage::Flush() at ??:0
@ 0x7f791b852a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f7931284512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f7931281976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f7931309610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f7931309439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f791e49d345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f791e4a091c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f791e4a0799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f791e47beaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f79217556db start_thread at ??:0
@ 0x7f791a6eb61f clone at ??:0
W20260812 07:05:48.261080 7089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:48.261353 7089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:48.261464 7089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:48.268563 7089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:48.268661 7089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:48.268782 7089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:05:48.278213 7089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:43141
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=43697
--tserver_master_addrs=127.11.192.254:33305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:48.279318 7089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:48.280864 7089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:05:48.293885 7164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:48.293979 7155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:48.293951 7156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:48.295128 7089 server_base.cc:1034] running on GCE node
I20260812 07:05:48.295697 7089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:48.296638 7089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:48.297844 7089 hybrid_clock.cc:648] HybridClock initialized: now 1786518348297788 us; error 45 us; skew 500 ppm
I20260812 07:05:48.298146 7089 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:48 dist-test-slave-lgr5 krb5kdc[4960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518348, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:48.301389 7089 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:05:48.302835 7089 webserver.cc:460] Webserver started at http://127.11.192.193:43697/ using document root <none> and password file <none>
I20260812 07:05:48.303498 7089 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:48.303632 7089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:05:48.308874 7089 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:05:48.311517 7175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:48.312758 7089 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:05:48.312919 7089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "6808c4ba75bb4745bb6bafada929a5ff"
format_stamp: "Formatted at 2026-08-12 07:05:42 on dist-test-slave-lgr5"
server_key: "e16011c21a5c65ac0ad160fff8b99161"
server_key_iv: "f4ddf39f064812a829e49f7401e36b35"
server_key_version: "encryptionkey@0"
I20260812 07:05:48.313828 7089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:48.327705 7089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:48.331501 7089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:48.331748 7089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:48.332614 7089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:05:48.334263 7184 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260812 07:05:48.339358 7184 ts_tablet_manager.cc:1852] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260812 07:05:48.340436 7184 ts_tablet_manager.cc:1905] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:05:48.347299 7184 ts_tablet_manager.cc:1918] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:05:48.347468 7184 log.cc:1196] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/wal/wals/88f81f66fc6c4388956a559829bf1489
I20260812 07:05:48.347909 7184 ts_tablet_manager.cc:1939] T 88f81f66fc6c4388956a559829bf1489 P 6808c4ba75bb4745bb6bafada929a5ff: Deleting consensus metadata
I20260812 07:05:48.348286 7089 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260812 07:05:48.348414 7089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.014s user 0.001s sys 0.000s
I20260812 07:05:48.348533 7089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:05:48.348614 7089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:48.379550 7089 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:43141
I20260812 07:05:48.379614 7363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:43141 every 8 connection(s)
I20260812 07:05:48.380833 7089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:05:48.385355 7368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:48.383569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.193:47543)
0812 07:05:48.384302 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:48.384543 (+ 241us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":267,"threads_started":1}
W20260812 07:05:48.385918 7366 heartbeater.cc:644] Failed to heartbeat to 127.11.192.254:33305 (0 consecutive failures): Network error: Failed to ping master at 127.11.192.254:33305: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
I20260812 07:05:48.386705 7368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:48.386344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.193:45803)
0812 07:05:48.386495 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:48.386582 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 07:05:48.387049 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 7089
I20260812 07:05:48.387641 7368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:48.387262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.193:49819)
0812 07:05:48.387401 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:48.387495 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 07:05:48.387864 7366 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 07:05:48.387849 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:45927
--local_ip_for_outbound_sockets=127.11.192.194
--tserver_master_addrs=127.11.192.254:33305
--webserver_port=36067
--webserver_interface=127.11.192.194
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:05:48.710862 7371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:48.711117 7371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:48.711248 7371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:48.718645 7371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:48.718761 7371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:48.718906 7371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:05:48.727999 7371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:45927
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=36067
--tserver_master_addrs=127.11.192.254:33305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:48.729158 7371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:48.730654 7371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:05:48.743932 7386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:48.743997 7388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:48.744048 7384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:48.744755 7371 server_base.cc:1034] running on GCE node
I20260812 07:05:48.745272 7371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:48.746125 7371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:48.747301 7371 hybrid_clock.cc:648] HybridClock initialized: now 1786518348747252 us; error 44 us; skew 500 ppm
I20260812 07:05:48.747594 7371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:48 dist-test-slave-lgr5 krb5kdc[4960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518348, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:48.750579 7371 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:05:48.752022 7371 webserver.cc:460] Webserver started at http://127.11.192.194:36067/ using document root <none> and password file <none>
I20260812 07:05:48.752691 7371 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:48.752808 7371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:05:48.757867 7371 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 07:05:48.760408 7398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:48.761607 7371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:05:48.761761 7371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "8c98955033894142b2ce660be9f2417d"
format_stamp: "Formatted at 2026-08-12 07:05:43 on dist-test-slave-lgr5"
server_key: "5a0750f971e6f0e23c4fdf37a6acabfb"
server_key_iv: "6e0e64a84750a0738841d3ff384ab084"
server_key_version: "encryptionkey@0"
I20260812 07:05:48.762693 7371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:48.778095 7371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:48.782006 7371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:48.782238 7371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:48.783109 7371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:05:48.784368 7371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:05:48.784459 7371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:48.784564 7371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:05:48.784638 7371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:48.814666 7371 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:45927
I20260812 07:05:48.814750 7610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:45927 every 8 connection(s)
I20260812 07:05:48.816040 7371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:05:48.820734 7615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:48.818996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.194:52851)
0812 07:05:48.819687 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:48.819891 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":272,"threads_started":1}
W20260812 07:05:48.821403 7613 heartbeater.cc:644] Failed to heartbeat to 127.11.192.254:33305 (0 consecutive failures): Network error: Failed to ping master at 127.11.192.254:33305: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
I20260812 07:05:48.822117 7615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:48.821832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.194:49931)
0812 07:05:48.821958 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:48.822020 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 07:05:48.822844 7615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:48.822557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.194:43887)
0812 07:05:48.822659 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:48.822731 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 07:05:48.823040 7613 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 07:05:48.825110 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 7371
I20260812 07:05:48.825675 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:33705
--local_ip_for_outbound_sockets=127.11.192.195
--tserver_master_addrs=127.11.192.254:33305
--webserver_port=46317
--webserver_interface=127.11.192.195
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:43473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:05:49.143424 7616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:05:49.143661 7616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:05:49.143787 7616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:05:49.150679 7616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:05:49.150771 7616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:05:49.150931 7616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:05:49.159994 7616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:33705
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=46317
--tserver_master_addrs=127.11.192.254:33305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:05:49.161126 7616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:05:49.162333 7616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:05:49.174630 7627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:49.174672 7628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:05:49.174702 7630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:05:49.175844 7616 server_base.cc:1034] running on GCE node
I20260812 07:05:49.176617 7616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:05:49.177497 7616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:05:49.178726 7616 hybrid_clock.cc:648] HybridClock initialized: now 1786518349178679 us; error 37 us; skew 500 ppm
I20260812 07:05:49.179047 7616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:05:49 dist-test-slave-lgr5 krb5kdc[4960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518349, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:05:49.182080 7616 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:05:49.183463 7616 webserver.cc:460] Webserver started at http://127.11.192.195:46317/ using document root <none> and password file <none>
I20260812 07:05:49.184151 7616 fs_manager.cc:359] Metadata directory not provided
I20260812 07:05:49.184276 7616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:05:49.189308 7616 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:05:49.191841 7647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:49.193107 7616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:05:49.193256 7616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "8db96391b7384bbf983337c330332568"
format_stamp: "Formatted at 2026-08-12 07:05:43 on dist-test-slave-lgr5"
server_key: "1ec9dfbf108e524251d730a2bd9ca28b"
server_key_iv: "b4bedc7c48b21f015a77a8c4fad05a27"
server_key_version: "encryptionkey@0"
I20260812 07:05:49.194175 7616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:05:49.217370 7616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:05:49.220860 7616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:05:49.221107 7616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:05:49.222033 7616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:05:49.223316 7616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:05:49.223426 7616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:49.223531 7616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:05:49.223605 7616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:05:49.253772 7616 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:33705
I20260812 07:05:49.253834 7830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:33705 every 8 connection(s)
I20260812 07:05:49.255164 7616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:05:49.259680 7837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:49.257940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.195:51533)
0812 07:05:49.258633 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:49.258863 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":260,"threads_started":1}
W20260812 07:05:49.260344 7834 heartbeater.cc:644] Failed to heartbeat to 127.11.192.254:33305 (0 consecutive failures): Network error: Failed to ping master at 127.11.192.254:33305: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
I20260812 07:05:49.260779 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 7616
I20260812 07:05:49.261184 7837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:49.260862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.195:46901)
0812 07:05:49.261021 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:49.261085 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 07:05:49.262018 7837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:49.261674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.195:32805)
0812 07:05:49.261785 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:49.261862 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 07:05:49.262256 7834 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 07:05:49.389051 7839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:49.388330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.193:38729)
0812 07:05:49.388746 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:49.388830 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":195,"threads_started":1}
I20260812 07:05:49.824313 7840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:49.823504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.194:54045)
0812 07:05:49.823945 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:49.824099 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":218,"threads_started":1}
I20260812 07:05:50.263523 7841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:50.262729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.195:40125)
0812 07:05:50.263195 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:50.263310 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":231,"threads_started":1}
I20260812 07:05:50.390487 7842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:50.389825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.193:60781)
0812 07:05:50.390259 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:50.390344 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":232,"threads_started":1}
I20260812 07:05:50.825803 7844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:50.825105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.194:50309)
0812 07:05:50.825567 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:50.825647 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":225,"threads_started":1}
I20260812 07:05:51.264981 7845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:51.264306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.195:53419)
0812 07:05:51.264746 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:51.264824 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":214,"threads_started":1}
I20260812 07:05:51.392114 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:51.391285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.193:55015)
0812 07:05:51.391826 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:51.391915 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":277,"threads_started":1}
I20260812 07:05:51.827234 7847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:51.826572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.194:50913)
0812 07:05:51.826989 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:51.827070 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":215,"threads_started":1}
I20260812 07:05:52.266496 7850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:52.265783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.195:50647)
0812 07:05:52.266213 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:52.266315 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":241,"threads_started":1}
I20260812 07:05:52.393520 7851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:52.392853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.193:54889)
0812 07:05:52.393262 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:52.393339 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":201,"threads_started":1}
I20260812 07:05:52.828696 7852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:52.827981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.194:45625)
0812 07:05:52.828420 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:52.828510 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":210,"threads_started":1}
I20260812 07:05:53.267959 7853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:53.267269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.195:57115)
0812 07:05:53.267690 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:53.267769 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":202,"threads_started":1}
I20260812 07:05:53.394944 7854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:53.394279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.193:33529)
0812 07:05:53.394711 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:53.394788 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":218,"threads_started":1}
I20260812 07:05:53.830235 7855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:53.829493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.194:35415)
0812 07:05:53.829989 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:53.830071 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":261,"threads_started":1}
I20260812 07:05:54.269480 7857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:54.268782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.195:60887)
0812 07:05:54.269248 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:54.269324 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":253,"threads_started":1}
I20260812 07:05:54.396462 7858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:54.395755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.193:60305)
0812 07:05:54.396226 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:54.396308 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":224,"threads_started":1}
I20260812 07:05:54.831938 7861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:54.831001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.194:37697)
0812 07:05:54.831645 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:54.831742 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":396,"threads_started":1}
I20260812 07:05:55.270905 7862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:55.270233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.195:55643)
0812 07:05:55.270665 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:55.270753 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":210,"threads_started":1}
I20260812 07:05:55.398154 7863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:55.397277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.193:59469)
0812 07:05:55.397858 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:55.397939 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":302,"threads_started":1}
I20260812 07:05:55.833508 7864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:55.832745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.194:50289)
0812 07:05:55.833214 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:55.833290 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":207,"threads_started":1}
I20260812 07:05:56.272557 7865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:56.271673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.195:55045)
0812 07:05:56.272253 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:56.272342 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":259,"threads_started":1}
I20260812 07:05:56.399565 7866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:56.398912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.193:55799)
0812 07:05:56.399326 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:56.399404 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":226,"threads_started":1}
I20260812 07:05:56.834952 7868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:56.834269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.194:39069)
0812 07:05:56.834712 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:56.834792 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":230,"threads_started":1}
I20260812 07:05:57.273999 7872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:57.273326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.195:41575)
0812 07:05:57.273754 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:57.273832 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":220,"threads_started":1}
I20260812 07:05:57.401149 7873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:57.400326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.193:53055)
0812 07:05:57.400829 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:57.400953 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":240,"threads_started":1}
I20260812 07:05:57.836490 7878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:57.835702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.194:60517)
0812 07:05:57.836207 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:57.836291 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":223,"threads_started":1}
I20260812 07:05:58.275455 7882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:58.274743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.195:35523)
0812 07:05:58.275199 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:58.275278 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":234,"threads_started":1}
I20260812 07:05:58.402634 7884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:58.401964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.193:38501)
0812 07:05:58.402393 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:58.402473 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":220,"threads_started":1}
I20260812 07:05:58.837916 7885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:58.837261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.194:53781)
0812 07:05:58.837686 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:58.837766 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":223,"threads_started":1}
I20260812 07:05:59.276944 7886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:59.276267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.195:40075)
0812 07:05:59.276706 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:59.276783 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":227,"threads_started":1}
I20260812 07:05:59.404074 7888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:59.403394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.193:43583)
0812 07:05:59.403816 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:59.403892 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":236,"threads_started":1}
I20260812 07:05:59.839401 7889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:05:59.838675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.194:46079)
0812 07:05:59.839141 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0812 07:05:59.839230 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":214,"threads_started":1}
I20260812 07:06:00.278404 7890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:00.277739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.195:38191)
0812 07:06:00.278139 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:00.278232 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":197,"threads_started":1}
I20260812 07:06:00.405488 7892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:00.404847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.193:52845)
0812 07:06:00.405262 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:00.405340 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":215,"threads_started":1}
I20260812 07:06:00.840860 7895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:00.840228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.194:43933)
0812 07:06:00.840637 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:00.840715 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":196,"threads_started":1}
I20260812 07:06:01.279872 7897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:01.279195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.195:50337)
0812 07:06:01.279640 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:01.279718 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":228,"threads_started":1}
I20260812 07:06:01.406968 7898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:01.406268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.193:38957)
0812 07:06:01.406687 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:01.406783 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":214,"threads_started":1}
I20260812 07:06:01.842347 7899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:01.841642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.194:41893)
0812 07:06:01.842077 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:01.842165 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":201,"threads_started":1}
I20260812 07:06:02.281451 7903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:02.280713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.195:58469)
0812 07:06:02.281175 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:02.281264 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":218,"threads_started":1}
I20260812 07:06:02.408422 7904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:02.407714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.193:58777)
0812 07:06:02.408163 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:02.408248 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":205,"threads_started":1}
I20260812 07:06:02.843767 7907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:02.843102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.194:49275)
0812 07:06:02.843508 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:02.843594 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":204,"threads_started":1}
I20260812 07:06:03.282925 7912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:03.282226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.195:54109)
0812 07:06:03.282671 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:03.282750 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":226,"threads_started":1}
I20260812 07:06:03.410084 7913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:03.409376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.193:52329)
0812 07:06:03.409797 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:03.409908 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":215,"threads_started":1}
I20260812 07:06:03.845419 7914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:03.844728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.194:43097)
0812 07:06:03.845135 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:03.845229 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":198,"threads_started":1}
I20260812 07:06:04.269446 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 7089
I20260812 07:06:04.284653 7919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:04.283871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:33305 (local address 127.11.192.195:46059)
0812 07:06:04.284364 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:04.284478 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:33305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":283,"threads_started":1}
I20260812 07:06:04.291568 7916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:04.270421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49280 (local address 127.11.192.193:43141)
0812 07:06:04.271111 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:04.271126 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:04.271167 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:06:04.273317 (+ 2150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:04.273328 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:04.273763 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:04.274065 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:06:04.275362 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:04.276687 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:04.277869 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:04.278518 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:04.278649 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:04.286191 (+ 7542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:06:04.286198 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:06:04.286213 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:06:04.286256 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:06:04.288947 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:04.289516 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:04.289522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:04.289529 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:04.289692 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:04.289968 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:04.289970 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:04.289972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:04.290474 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:06:04.290714 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:04.290977 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 07:06:04.291287 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":336,"threads_started":1}
I20260812 07:06:04.292922 7358 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:06:04.377812 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 7089
I20260812 07:06:04.403918 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 7371
I20260812 07:06:04.420540 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:04.404819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58952 (local address 127.11.192.194:45927)
0812 07:06:04.405342 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:04.405356 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:04.405393 (+ 37us) server_negotiation.cc:407] Connection header received
0812 07:06:04.406998 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:04.407008 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:04.407420 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:04.407712 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:06:04.409045 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:04.410256 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:04.411411 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:04.412044 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:04.412242 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:04.414813 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:06:04.414821 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:06:04.414835 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:06:04.414876 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:06:04.417418 (+ 2542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:04.418105 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:04.418112 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:04.418118 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:04.418280 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:04.418641 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:04.418645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:04.418647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:04.419128 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:06:04.419429 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:04.419844 (+ 415us) server_negotiation.cc:299] Negotiation successful
0812 07:06:04.420255 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":229,"threads_started":1}
I20260812 07:06:04.421836 7605 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:06:04.506186 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 7371
I20260812 07:06:04.533530 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 7616
I20260812 07:06:04.550019 7929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:04.534597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39012 (local address 127.11.192.195:33705)
0812 07:06:04.535232 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:04.535246 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:04.535289 (+ 43us) server_negotiation.cc:407] Connection header received
0812 07:06:04.536960 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:04.536969 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:04.537384 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:04.537665 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:06:04.538937 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:04.540228 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:04.541391 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:04.542024 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:04.542157 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:04.544592 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:06:04.544600 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:06:04.544613 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:06:04.544663 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:06:04.547246 (+ 2583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:04.547823 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:04.547829 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:04.547834 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:04.547988 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:04.548331 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:04.548334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:04.548336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:04.548827 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:06:04.549069 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:04.549353 (+ 284us) server_negotiation.cc:299] Negotiation successful
0812 07:06:04.549727 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":294,"threads_started":1}
I20260812 07:06:04.551291 7824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:06:04.633584 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 7616
2026-08-12T07:06:04Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24288 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:06:04 dist-test-slave-lgr5 krb5kdc[7935](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:06:04 dist-test-slave-lgr5 krb5kdc[7935](info): set up 2 sockets
Aug 12 07:06:04 dist-test-slave-lgr5 krb5kdc[7935](info): commencing operation
krb5kdc: starting...
W20260812 07:06:05.886245 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:06:06 dist-test-slave-lgr5 krb5kdc[7935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:06:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:06:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:06:06.127838 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:37881
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:37881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:06:06.450304 7961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:06:06.450558 7961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:06:06.450650 7961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:06:06.457526 7961 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 07:06:06.457608 7961 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 07:06:06.457651 7961 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 07:06:06.457701 7961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:06:06.457758 7961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:06:06.457821 7961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:06:06.457866 7961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:06:06.457918 7961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:06:06.468712 7961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:37881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:37881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:06:06.469892 7961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:06:06.471170 7961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:06:06.483016 7971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:06.483230 7966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:06.483016 7968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:06:06.483695 7961 server_base.cc:1034] running on GCE node
I20260812 07:06:06.484504 7961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:06:06.485823 7961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:06:06.487048 7961 hybrid_clock.cc:648] HybridClock initialized: now 1786518366486991 us; error 26 us; skew 500 ppm
I20260812 07:06:06.487362 7961 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:06:06 dist-test-slave-lgr5 krb5kdc[7935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518366, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:06:06.490506 7961 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:06:06.491950 7961 webserver.cc:460] Webserver started at http://127.11.192.254:46593/ using document root <none> and password file <none>
I20260812 07:06:06.492643 7961 fs_manager.cc:359] Metadata directory not provided
I20260812 07:06:06.492741 7961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:06:06.493024 7961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:06:06.495959 7961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "4ee74e07b85b420d9a3faea5221e66a4"
format_stamp: "Formatted at 2026-08-12 07:06:06 on dist-test-slave-lgr5"
server_key: "f10d94cc9bbf70796bd69cd8d36a8bff"
server_key_iv: "a485227446a83ca1386e99541ff35f0c"
server_key_version: "encryptionkey@0"
I20260812 07:06:06.496819 7961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "4ee74e07b85b420d9a3faea5221e66a4"
format_stamp: "Formatted at 2026-08-12 07:06:06 on dist-test-slave-lgr5"
server_key: "f10d94cc9bbf70796bd69cd8d36a8bff"
server_key_iv: "a485227446a83ca1386e99541ff35f0c"
server_key_version: "encryptionkey@0"
I20260812 07:06:06.501873 7961 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:06:06.505208 7979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:06.506469 7961 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:06:06.506642 7961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "4ee74e07b85b420d9a3faea5221e66a4"
format_stamp: "Formatted at 2026-08-12 07:06:06 on dist-test-slave-lgr5"
server_key: "f10d94cc9bbf70796bd69cd8d36a8bff"
server_key_iv: "a485227446a83ca1386e99541ff35f0c"
server_key_version: "encryptionkey@0"
I20260812 07:06:06.506844 7961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:06:06.518224 7961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:06:06.521863 7961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:06:06.522100 7961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:06:06.540486 7961 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:37881
I20260812 07:06:06.540474 8060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:37881 every 8 connection(s)
I20260812 07:06:06.541801 7961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:06:06.545182 8064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:06:06.547554 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 7961
I20260812 07:06:06.547763 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:06:06.548633 12035 external_mini_cluster.cc:1428] Setting key db27bee6b1955a5341fcb6f2f940a1d5
I20260812 07:06:06.556397 8064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4: Bootstrap starting.
I20260812 07:06:06.559165 8064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4: Neither blocks nor log segments found. Creating new log.
I20260812 07:06:06.560199 8064 log.cc:824] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4: Log is configured to *not* fsync() on all Append() calls
Aug 12 07:06:06 dist-test-slave-lgr5 krb5kdc[7935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:06:06.563200 8064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4: No bootstrap required, opened a new log
I20260812 07:06:06.571557 8072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:06.550704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37150 (local address 127.11.192.254:37881)
0812 07:06:06.551486 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:06.551497 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:06.551538 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:06:06.553290 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:06.553317 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:06.553690 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:06.554064 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:06:06.555725 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:06.557136 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:06.558544 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:06.559223 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:06.559451 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:06.564450 (+ 4999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:06:06.564459 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:06:06.564472 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:06:06.564509 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:06:06.567694 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:06.568553 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:06.568560 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:06.568566 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:06.568752 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:06.569147 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:06.569150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:06.569152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:06.569628 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:06:06.569863 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:06.570301 (+ 438us) server_negotiation.cc:299] Negotiation successful
0812 07:06:06.570614 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":354,"threads_started":1}
I20260812 07:06:06.571818 8064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee74e07b85b420d9a3faea5221e66a4" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 37881 } }
I20260812 07:06:06.572192 8064 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:06:06.572276 8064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee74e07b85b420d9a3faea5221e66a4, State: Initialized, Role: FOLLOWER
I20260812 07:06:06.572809 8064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee74e07b85b420d9a3faea5221e66a4" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 37881 } }
I20260812 07:06:06.573043 8064 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:06:06.573176 8064 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:06:06.573328 8064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:06:06.575388 8064 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee74e07b85b420d9a3faea5221e66a4" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 37881 } }
I20260812 07:06:06.575829 8064 leader_election.cc:302] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ee74e07b85b420d9a3faea5221e66a4; no voters:
I20260812 07:06:06.576399 8064 leader_election.cc:288] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:06:06.576519 8074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:06:06.577126 8074 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 1 LEADER]: Becoming Leader. State: Replica: 4ee74e07b85b420d9a3faea5221e66a4, State: Running, Role: LEADER
I20260812 07:06:06.577710 8074 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee74e07b85b420d9a3faea5221e66a4" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 37881 } }
I20260812 07:06:06.578215 8064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:06:06.580255 8078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ee74e07b85b420d9a3faea5221e66a4. Latest consensus state: current_term: 1 leader_uuid: "4ee74e07b85b420d9a3faea5221e66a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee74e07b85b420d9a3faea5221e66a4" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 37881 } } }
I20260812 07:06:06.580343 8076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ee74e07b85b420d9a3faea5221e66a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee74e07b85b420d9a3faea5221e66a4" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 37881 } } }
I20260812 07:06:06.580593 8078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [sys.catalog]: This master's current role is: LEADER
I20260812 07:06:06.580725 8076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [sys.catalog]: This master's current role is: LEADER
I20260812 07:06:06.581483 8083 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:06:06.586702 8083 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:06:06.594548 8083 catalog_manager.cc:1347] Generated new cluster ID: 8f2470c482394ec7b633fbb5816820f0
I20260812 07:06:06.594645 8083 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:06:06.602676 8083 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:06:06.603811 8083 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:06:06.609359 8083 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4: Generated new TSK 0
I20260812 07:06:06.610061 8083 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
I20260812 07:06:06.671001 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:37881
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:06:07.007320 8108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:06:07.007608 8108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:06:07.007751 8108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:06:07.015017 8108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:06:07.015142 8108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:06:07.015296 8108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:06:07.024698 8108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:37881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:06:07.025969 8108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:06:07.027251 8108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:06:07.039422 8120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:07.039477 8124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:07.039451 8122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:06:07.040616 8108 server_base.cc:1034] running on GCE node
I20260812 07:06:07.041148 8108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:06:07.041965 8108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:06:07.043165 8108 hybrid_clock.cc:648] HybridClock initialized: now 1786518367043127 us; error 31 us; skew 500 ppm
I20260812 07:06:07.043431 8108 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:06:07 dist-test-slave-lgr5 krb5kdc[7935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518367, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:06:07.046370 8108 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:06:07.047679 8108 webserver.cc:460] Webserver started at http://127.11.192.193:41117/ using document root <none> and password file <none>
I20260812 07:06:07.048345 8108 fs_manager.cc:359] Metadata directory not provided
I20260812 07:06:07.048460 8108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:06:07.048759 8108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:06:07.051544 8108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "2fd1bd31c2b44db8af74961163f0f889"
format_stamp: "Formatted at 2026-08-12 07:06:07 on dist-test-slave-lgr5"
server_key: "6c62a0211db307d9bc49b8b898fb0f9c"
server_key_iv: "a528c1d510c621f597de99da4b75fd6f"
server_key_version: "encryptionkey@0"
I20260812 07:06:07.052327 8108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "2fd1bd31c2b44db8af74961163f0f889"
format_stamp: "Formatted at 2026-08-12 07:06:07 on dist-test-slave-lgr5"
server_key: "6c62a0211db307d9bc49b8b898fb0f9c"
server_key_iv: "a528c1d510c621f597de99da4b75fd6f"
server_key_version: "encryptionkey@0"
I20260812 07:06:07.057180 8108 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:06:07.060330 8136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:07.061545 8108 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260812 07:06:07.061684 8108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "2fd1bd31c2b44db8af74961163f0f889"
format_stamp: "Formatted at 2026-08-12 07:06:07 on dist-test-slave-lgr5"
server_key: "6c62a0211db307d9bc49b8b898fb0f9c"
server_key_iv: "a528c1d510c621f597de99da4b75fd6f"
server_key_version: "encryptionkey@0"
I20260812 07:06:07.061851 8108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:06:07.071839 8108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:06:07.075776 8108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:06:07.076030 8108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:06:07.076915 8108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:06:07.078148 8108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:06:07.078235 8108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:07.078352 8108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:06:07.078408 8108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:07.108105 8108 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:41097
I20260812 07:06:07.108165 8327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:41097 every 8 connection(s)
I20260812 07:06:07.109378 8108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:06:07.114564 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 8108
I20260812 07:06:07.114777 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:06:07.115628 12035 external_mini_cluster.cc:1428] Setting key 46488a0b37992df396639292b2d125b6
Aug 12 07:06:07 dist-test-slave-lgr5 krb5kdc[7935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518367, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:06:07.130098 8334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:07.111787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:46927 (local address 127.11.192.254:37881)
0812 07:06:07.112254 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:07.112260 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:07.113188 (+ 928us) server_negotiation.cc:407] Connection header received
0812 07:06:07.115322 (+ 2134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:07.115326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:07.115394 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:07.115494 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:06:07.117308 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:07.118140 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:07.120246 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:07.120695 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:07.120827 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:07.124704 (+ 3877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:06:07.124709 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:06:07.124712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:06:07.124744 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:06:07.127193 (+ 2449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:07.127950 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:07.127953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:07.127955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:07.128099 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:07.128529 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:07.128533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:07.128535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:07.128784 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:06:07.128892 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:07.129622 (+ 730us) server_negotiation.cc:299] Negotiation successful
0812 07:06:07.129785 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":205,"threads_started":1}
I20260812 07:06:07.131080 8338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:07.112355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.193:46927)
0812 07:06:07.113030 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:07.113076 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:06:07.115008 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:06:07.115757 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:06:07.115768 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:06:07.116474 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:06:07.117143 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:07.117162 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:07.118305 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:07.118312 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:07.120118 (+ 1806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:07.120135 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:07.120975 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:07.120984 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:07.121294 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:07.121945 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:06:07.121971 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:06:07.124487 (+ 2516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:06:07.127328 (+ 2841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:06:07.127336 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:06:07.127350 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:06:07.127848 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:06:07.128198 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:06:07.128202 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:06:07.128205 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:06:07.128433 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:06:07.128904 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:06:07.128912 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:06:07.129276 (+ 364us) client_negotiation.cc:769] Sending connection context
0812 07:06:07.129545 (+ 269us) client_negotiation.cc:240] Negotiation successful
0812 07:06:07.130040 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":258,"threads_started":1}
I20260812 07:06:07.132542 8328 heartbeater.cc:347] Connected to a master server at 127.11.192.254:37881
I20260812 07:06:07.132853 8328 heartbeater.cc:464] Registering TS with master...
I20260812 07:06:07.133687 8328 heartbeater.cc:511] Master 127.11.192.254:37881 requested a full tablet report, sending...
I20260812 07:06:07.135737 8004 ts_manager.cc:194] Registered new tserver with Master: 2fd1bd31c2b44db8af74961163f0f889 (127.11.192.193:41097)
I20260812 07:06:07.137169 8004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:46927
WARNING: no policy specified for kudu/127.11.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
I20260812 07:06:07.177477 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:37881
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:06:07.506426 8346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:06:07.506675 8346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:06:07.506803 8346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:06:07.513680 8346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:06:07.513774 8346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:06:07.513919 8346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:06:07.522996 8346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:37881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:06:07.524135 8346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:06:07.525343 8346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:06:07.537762 8357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:07.537909 8355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:07.538213 8359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:06:07.538550 8346 server_base.cc:1034] running on GCE node
I20260812 07:06:07.539074 8346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:06:07.539913 8346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:06:07.541179 8346 hybrid_clock.cc:648] HybridClock initialized: now 1786518367541116 us; error 48 us; skew 500 ppm
I20260812 07:06:07.541481 8346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:06:07 dist-test-slave-lgr5 krb5kdc[7935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518367, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:06:07.544479 8346 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:06:07.545821 8346 webserver.cc:460] Webserver started at http://127.11.192.194:46083/ using document root <none> and password file <none>
I20260812 07:06:07.546446 8346 fs_manager.cc:359] Metadata directory not provided
I20260812 07:06:07.546550 8346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:06:07.546826 8346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:06:07.549568 8346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "149bf9df73814bbabf72c34058df1ed1"
format_stamp: "Formatted at 2026-08-12 07:06:07 on dist-test-slave-lgr5"
server_key: "71c6818c8664e1c9f1d62c15c87dabc4"
server_key_iv: "b40201d1784c17dc0e4c8f7f140bcb6a"
server_key_version: "encryptionkey@0"
I20260812 07:06:07.550272 8346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "149bf9df73814bbabf72c34058df1ed1"
format_stamp: "Formatted at 2026-08-12 07:06:07 on dist-test-slave-lgr5"
server_key: "71c6818c8664e1c9f1d62c15c87dabc4"
server_key_iv: "b40201d1784c17dc0e4c8f7f140bcb6a"
server_key_version: "encryptionkey@0"
I20260812 07:06:07.554937 8346 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 07:06:07.558167 8373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:07.559324 8346 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 07:06:07.559479 8346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "149bf9df73814bbabf72c34058df1ed1"
format_stamp: "Formatted at 2026-08-12 07:06:07 on dist-test-slave-lgr5"
server_key: "71c6818c8664e1c9f1d62c15c87dabc4"
server_key_iv: "b40201d1784c17dc0e4c8f7f140bcb6a"
server_key_version: "encryptionkey@0"
I20260812 07:06:07.559670 8346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:06:07.582757 8346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:06:07.586360 8346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:06:07.586612 8346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:06:07.587474 8346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:06:07.588770 8346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:06:07.588843 8346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:07.588949 8346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:06:07.588999 8346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:07.618909 8346 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:38967
I20260812 07:06:07.618966 8591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:38967 every 8 connection(s)
I20260812 07:06:07.620097 8346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:06:07.620687 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 8346
I20260812 07:06:07.620878 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:06:07.621475 12035 external_mini_cluster.cc:1428] Setting key 5becaba6ac4ecbe3dbfc063fe25781ee
Aug 12 07:06:07 dist-test-slave-lgr5 krb5kdc[7935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518367, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:06:07.639240 8334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:07.622633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:59079 (local address 127.11.192.254:37881)
0812 07:06:07.622812 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:07.622818 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:07.624056 (+ 1238us) server_negotiation.cc:407] Connection header received
0812 07:06:07.625862 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:07.625867 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:07.625941 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:07.626035 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:06:07.627406 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:07.628332 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:07.629787 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:07.630359 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:07.630483 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:07.633709 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:06:07.633715 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:06:07.633719 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:06:07.633756 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:06:07.636406 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:07.637072 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:07.637078 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:07.637082 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:07.637236 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:07.637631 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:07.637636 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:07.637639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:07.637967 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:06:07.638125 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:07.638709 (+ 584us) server_negotiation.cc:299] Negotiation successful
0812 07:06:07.638927 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:06:07.639948 8597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:07.623069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.194:59079)
0812 07:06:07.623869 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:07.623921 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 07:06:07.625610 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:06:07.626224 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:06:07.626231 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:06:07.626691 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:06:07.627235 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:07.627253 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:07.628464 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:07.628468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:07.629644 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:07.629656 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:07.630543 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:07.630547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:07.630735 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:07.631322 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:06:07.631344 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:06:07.633504 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:06:07.636532 (+ 3028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:06:07.636541 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:06:07.636555 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:06:07.636957 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:06:07.637344 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:06:07.637348 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:06:07.637350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:06:07.637532 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:06:07.638080 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:06:07.638085 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:06:07.638360 (+ 275us) client_negotiation.cc:769] Sending connection context
0812 07:06:07.638577 (+ 217us) client_negotiation.cc:240] Negotiation successful
0812 07:06:07.638932 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":330,"threads_started":1}
I20260812 07:06:07.641269 8594 heartbeater.cc:347] Connected to a master server at 127.11.192.254:37881
I20260812 07:06:07.641624 8594 heartbeater.cc:464] Registering TS with master...
I20260812 07:06:07.642402 8594 heartbeater.cc:511] Master 127.11.192.254:37881 requested a full tablet report, sending...
I20260812 07:06:07.643877 8004 ts_manager.cc:194] Registered new tserver with Master: 149bf9df73814bbabf72c34058df1ed1 (127.11.192.194:38967)
I20260812 07:06:07.644681 8004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:59079
WARNING: no policy specified for kudu/127.11.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
I20260812 07:06:07.682655 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:37881
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:06:07.998682 8601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:06:07.998929 8601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:06:07.999064 8601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:06:08.006023 8601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:06:08.006124 8601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:06:08.006271 8601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:06:08.015707 8601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:37881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:06:08.016860 8601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:06:08.018056 8601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:06:08.030133 8611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:08.030067 8623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:08.030113 8613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:06:08.030922 8601 server_base.cc:1034] running on GCE node
I20260812 07:06:08.031420 8601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:06:08.032265 8601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:06:08.033461 8601 hybrid_clock.cc:648] HybridClock initialized: now 1786518368033408 us; error 42 us; skew 500 ppm
I20260812 07:06:08.033753 8601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:06:08 dist-test-slave-lgr5 krb5kdc[7935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518368, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:06:08.036679 8601 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:06:08.037946 8601 webserver.cc:460] Webserver started at http://127.11.192.195:36431/ using document root <none> and password file <none>
I20260812 07:06:08.038563 8601 fs_manager.cc:359] Metadata directory not provided
I20260812 07:06:08.038678 8601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:06:08.038960 8601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:06:08.041710 8601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "fa426e29ae2342c09714cbdeba8db23f"
format_stamp: "Formatted at 2026-08-12 07:06:08 on dist-test-slave-lgr5"
server_key: "9fb6cdf0a648a4ff9b3c03c008a135e3"
server_key_iv: "ecabdf9367798e0481d0a46251b5b0e7"
server_key_version: "encryptionkey@0"
I20260812 07:06:08.042465 8601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "fa426e29ae2342c09714cbdeba8db23f"
format_stamp: "Formatted at 2026-08-12 07:06:08 on dist-test-slave-lgr5"
server_key: "9fb6cdf0a648a4ff9b3c03c008a135e3"
server_key_iv: "ecabdf9367798e0481d0a46251b5b0e7"
server_key_version: "encryptionkey@0"
I20260812 07:06:08.047533 8601 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260812 07:06:08.050721 8636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:08.051864 8601 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:06:08.052014 8601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "fa426e29ae2342c09714cbdeba8db23f"
format_stamp: "Formatted at 2026-08-12 07:06:08 on dist-test-slave-lgr5"
server_key: "9fb6cdf0a648a4ff9b3c03c008a135e3"
server_key_iv: "ecabdf9367798e0481d0a46251b5b0e7"
server_key_version: "encryptionkey@0"
I20260812 07:06:08.052250 8601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:06:08.068195 8601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:06:08.071604 8601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:06:08.071846 8601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:06:08.072749 8601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:06:08.074011 8601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:06:08.074088 8601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:08.074208 8601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:06:08.074260 8601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:08.103415 8601 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:46369
I20260812 07:06:08.103454 8837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:46369 every 8 connection(s)
I20260812 07:06:08.104671 8601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:06:08.113502 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 8601
I20260812 07:06:08.113701 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:06:08.114279 12035 external_mini_cluster.cc:1428] Setting key b59ce7da8c628ed5b11629ea228b1fc9
Aug 12 07:06:08 dist-test-slave-lgr5 krb5kdc[7935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518368, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:06:08.124745 8334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:08.106942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:52565 (local address 127.11.192.254:37881)
0812 07:06:08.107160 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:08.107167 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:08.108412 (+ 1245us) server_negotiation.cc:407] Connection header received
0812 07:06:08.110869 (+ 2457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:08.110874 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:08.110968 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:08.111064 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:06:08.112921 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.113701 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.115678 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.116152 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.116250 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:08.120036 (+ 3786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:06:08.120041 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:06:08.120044 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:06:08.120098 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:06:08.122380 (+ 2282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:08.122986 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:08.122989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:08.122991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:08.123110 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:08.123433 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:08.123436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:08.123438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:08.123677 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:06:08.123783 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:08.124356 (+ 573us) server_negotiation.cc:299] Negotiation successful
0812 07:06:08.124511 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 07:06:08.125551 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:08.107390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.195:52565)
0812 07:06:08.108229 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:08.108287 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 07:06:08.110596 (+ 2309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:06:08.111321 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:06:08.111332 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:06:08.111993 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:06:08.112751 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:08.112779 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.113895 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.113901 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:08.115548 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:08.115561 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.116984 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.116995 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:08.117253 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:08.117832 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:06:08.117852 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:06:08.119858 (+ 2006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:06:08.122497 (+ 2639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:06:08.122503 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:06:08.122514 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:06:08.122890 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:06:08.123186 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:06:08.123189 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:06:08.123191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:06:08.123358 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:06:08.123763 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:06:08.123768 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:06:08.124032 (+ 264us) client_negotiation.cc:769] Sending connection context
0812 07:06:08.124274 (+ 242us) client_negotiation.cc:240] Negotiation successful
0812 07:06:08.124609 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":307,"threads_started":1}
I20260812 07:06:08.126725 8840 heartbeater.cc:347] Connected to a master server at 127.11.192.254:37881
I20260812 07:06:08.126981 8840 heartbeater.cc:464] Registering TS with master...
I20260812 07:06:08.127631 8840 heartbeater.cc:511] Master 127.11.192.254:37881 requested a full tablet report, sending...
I20260812 07:06:08.129225 8004 ts_manager.cc:194] Registered new tserver with Master: fa426e29ae2342c09714cbdeba8db23f (127.11.192.195:46369)
I20260812 07:06:08.130023 8004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:52565
I20260812 07:06:08.140384 8328 heartbeater.cc:503] Master 127.11.192.254:37881 was elected leader, sending a full tablet report...
I20260812 07:06:08.141438 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:06:08.157740 8334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:08.145863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37156 (local address 127.11.192.254:37881)
0812 07:06:08.146007 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:08.146011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:08.146477 (+ 466us) server_negotiation.cc:407] Connection header received
0812 07:06:08.146623 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:08.146629 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:08.146687 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:08.146764 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:06:08.148138 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.148815 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.150177 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.150577 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.150684 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:08.153076 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:06:08.153081 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:06:08.153084 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:06:08.153113 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:06:08.155250 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:08.155890 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:08.155896 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:08.155899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:08.156034 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:08.156397 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:08.156402 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:08.156405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:08.156705 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:06:08.156938 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:08.157261 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 07:06:08.157459 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 07:06:08.161336 8003 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:06:08.168162 8003 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:06:08.192670 8863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:08.179592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35584 (local address 127.11.192.195:46369)
0812 07:06:08.184543 (+ 4951us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:08.184551 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:08.184583 (+ 32us) server_negotiation.cc:407] Connection header received
0812 07:06:08.184681 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:08.184688 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:08.184970 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:08.185804 (+ 834us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:06:08.186125 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.187704 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.190153 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.191531 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.191709 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:08.191872 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:08.192184 (+ 312us) server_negotiation.cc:299] Negotiation successful
0812 07:06:08.192341 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4774,"thread_start_us":4613,"threads_started":1}
I20260812 07:06:08.192709 8859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:08.179158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:46369 (local address 127.0.0.1:35584)
0812 07:06:08.179875 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:08.179919 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 07:06:08.180102 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:06:08.185201 (+ 5099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:06:08.185208 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:06:08.185239 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:06:08.185609 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:08.185622 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.187836 (+ 2214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.187841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:08.189980 (+ 2139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:08.189996 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.191713 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.191719 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:08.191899 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:08.191950 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 07:06:08.192107 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 07:06:08.192278 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"mutex_wait_us":21,"thread_start_us":724,"threads_started":1}
I20260812 07:06:08.193243 8860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:08.179521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:41097 (local address 127.0.0.1:40024)
0812 07:06:08.184765 (+ 5244us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:08.184786 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:06:08.184925 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:06:08.186117 (+ 1192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:06:08.186124 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:06:08.186151 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:06:08.186567 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:08.186581 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.188613 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.188618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:08.190511 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:08.190522 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.192470 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.192475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:08.192633 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:08.192669 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 07:06:08.192740 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 07:06:08.192851 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5129,"mutex_wait_us":23,"thread_start_us":4984,"threads_started":1}
I20260812 07:06:08.194031 8868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:08.185812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:38967 (local address 127.0.0.1:46430)
0812 07:06:08.186330 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:08.186347 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:06:08.186486 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:06:08.187383 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:06:08.187387 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:06:08.187411 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:06:08.187709 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:08.187718 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.189245 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.189249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:08.191181 (+ 1932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:08.191196 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.193198 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.193205 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:08.193361 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:08.193393 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 07:06:08.193514 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 07:06:08.193631 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":254,"threads_started":1}
I20260812 07:06:08.194038 8867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:08.185208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46430 (local address 127.11.192.194:38967)
0812 07:06:08.185762 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:08.185770 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:08.186454 (+ 684us) server_negotiation.cc:407] Connection header received
0812 07:06:08.186698 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:08.186706 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:08.186924 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:08.187090 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:06:08.187901 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.189111 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.191364 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.192700 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.192844 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:08.192983 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:08.193567 (+ 584us) server_negotiation.cc:299] Negotiation successful
0812 07:06:08.193730 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":305,"threads_started":1}
I20260812 07:06:08.196033 8865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:08.179833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40024 (local address 127.11.192.193:41097)
0812 07:06:08.185489 (+ 5656us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:08.185497 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:08.185524 (+ 27us) server_negotiation.cc:407] Connection header received
0812 07:06:08.185617 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:08.185623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:08.185838 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:08.186764 (+ 926us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:06:08.186969 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.188478 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.190813 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.192245 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.193628 (+ 1383us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:08.193787 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:08.193853 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 07:06:08.194012 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5485,"thread_start_us":5343,"threads_started":1}
I20260812 07:06:08.196862 8716 tablet_service.cc:1467] Processing CreateTablet for tablet db6c323bbb1e4af099d3cd54d8499432 (DEFAULT_TABLE table=test-table [id=0893bf56af504868b0f128c22080e266]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:06:08.197606 8461 tablet_service.cc:1467] Processing CreateTablet for tablet db6c323bbb1e4af099d3cd54d8499432 (DEFAULT_TABLE table=test-table [id=0893bf56af504868b0f128c22080e266]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:06:08.198438 8716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db6c323bbb1e4af099d3cd54d8499432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:06:08.198943 8461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db6c323bbb1e4af099d3cd54d8499432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:06:08.198809 8213 tablet_service.cc:1467] Processing CreateTablet for tablet db6c323bbb1e4af099d3cd54d8499432 (DEFAULT_TABLE table=test-table [id=0893bf56af504868b0f128c22080e266]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:06:08.199874 8213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db6c323bbb1e4af099d3cd54d8499432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:06:08.210567 8869 tablet_bootstrap.cc:492] T db6c323bbb1e4af099d3cd54d8499432 P 2fd1bd31c2b44db8af74961163f0f889: Bootstrap starting.
I20260812 07:06:08.212594 8871 tablet_bootstrap.cc:492] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1: Bootstrap starting.
I20260812 07:06:08.213693 8870 tablet_bootstrap.cc:492] T db6c323bbb1e4af099d3cd54d8499432 P fa426e29ae2342c09714cbdeba8db23f: Bootstrap starting.
I20260812 07:06:08.214393 8869 tablet_bootstrap.cc:654] T db6c323bbb1e4af099d3cd54d8499432 P 2fd1bd31c2b44db8af74961163f0f889: Neither blocks nor log segments found. Creating new log.
I20260812 07:06:08.215709 8869 log.cc:824] T db6c323bbb1e4af099d3cd54d8499432 P 2fd1bd31c2b44db8af74961163f0f889: Log is configured to *not* fsync() on all Append() calls
I20260812 07:06:08.216945 8870 tablet_bootstrap.cc:654] T db6c323bbb1e4af099d3cd54d8499432 P fa426e29ae2342c09714cbdeba8db23f: Neither blocks nor log segments found. Creating new log.
I20260812 07:06:08.218127 8870 log.cc:824] T db6c323bbb1e4af099d3cd54d8499432 P fa426e29ae2342c09714cbdeba8db23f: Log is configured to *not* fsync() on all Append() calls
I20260812 07:06:08.218789 8869 tablet_bootstrap.cc:492] T db6c323bbb1e4af099d3cd54d8499432 P 2fd1bd31c2b44db8af74961163f0f889: No bootstrap required, opened a new log
I20260812 07:06:08.219053 8871 tablet_bootstrap.cc:654] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1: Neither blocks nor log segments found. Creating new log.
I20260812 07:06:08.219071 8869 ts_tablet_manager.cc:1397] T db6c323bbb1e4af099d3cd54d8499432 P 2fd1bd31c2b44db8af74961163f0f889: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 07:06:08.220260 8871 log.cc:824] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1: Log is configured to *not* fsync() on all Append() calls
I20260812 07:06:08.220499 8870 tablet_bootstrap.cc:492] T db6c323bbb1e4af099d3cd54d8499432 P fa426e29ae2342c09714cbdeba8db23f: No bootstrap required, opened a new log
I20260812 07:06:08.220729 8870 ts_tablet_manager.cc:1397] T db6c323bbb1e4af099d3cd54d8499432 P fa426e29ae2342c09714cbdeba8db23f: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.005s
I20260812 07:06:08.223097 8871 tablet_bootstrap.cc:492] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1: No bootstrap required, opened a new log
I20260812 07:06:08.223336 8871 ts_tablet_manager.cc:1397] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.006s
I20260812 07:06:08.228222 8870 raft_consensus.cc:348] T db6c323bbb1e4af099d3cd54d8499432 P fa426e29ae2342c09714cbdeba8db23f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } }
I20260812 07:06:08.228556 8870 raft_consensus.cc:374] T db6c323bbb1e4af099d3cd54d8499432 P fa426e29ae2342c09714cbdeba8db23f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:06:08.228657 8870 raft_consensus.cc:729] T db6c323bbb1e4af099d3cd54d8499432 P fa426e29ae2342c09714cbdeba8db23f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa426e29ae2342c09714cbdeba8db23f, State: Initialized, Role: FOLLOWER
I20260812 07:06:08.228775 8869 raft_consensus.cc:348] T db6c323bbb1e4af099d3cd54d8499432 P 2fd1bd31c2b44db8af74961163f0f889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } }
I20260812 07:06:08.229171 8869 raft_consensus.cc:374] T db6c323bbb1e4af099d3cd54d8499432 P 2fd1bd31c2b44db8af74961163f0f889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:06:08.229274 8869 raft_consensus.cc:729] T db6c323bbb1e4af099d3cd54d8499432 P 2fd1bd31c2b44db8af74961163f0f889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd1bd31c2b44db8af74961163f0f889, State: Initialized, Role: FOLLOWER
I20260812 07:06:08.229224 8870 consensus_queue.cc:260] T db6c323bbb1e4af099d3cd54d8499432 P fa426e29ae2342c09714cbdeba8db23f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } }
I20260812 07:06:08.229842 8869 consensus_queue.cc:260] T db6c323bbb1e4af099d3cd54d8499432 P 2fd1bd31c2b44db8af74961163f0f889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } }
I20260812 07:06:08.230283 8840 heartbeater.cc:503] Master 127.11.192.254:37881 was elected leader, sending a full tablet report...
I20260812 07:06:08.230876 8870 ts_tablet_manager.cc:1428] T db6c323bbb1e4af099d3cd54d8499432 P fa426e29ae2342c09714cbdeba8db23f: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260812 07:06:08.232285 8869 ts_tablet_manager.cc:1428] T db6c323bbb1e4af099d3cd54d8499432 P 2fd1bd31c2b44db8af74961163f0f889: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260812 07:06:08.233188 8871 raft_consensus.cc:348] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } }
I20260812 07:06:08.233480 8871 raft_consensus.cc:374] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:06:08.233568 8871 raft_consensus.cc:729] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 149bf9df73814bbabf72c34058df1ed1, State: Initialized, Role: FOLLOWER
I20260812 07:06:08.234086 8871 consensus_queue.cc:260] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } }
I20260812 07:06:08.235211 8594 heartbeater.cc:503] Master 127.11.192.254:37881 was elected leader, sending a full tablet report...
I20260812 07:06:08.235653 8871 ts_tablet_manager.cc:1428] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1: Time spent starting tablet: real 0.012s user 0.006s sys 0.007s
I20260812 07:06:08.327605 8881 raft_consensus.cc:482] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:06:08.327879 8881 raft_consensus.cc:504] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } }
I20260812 07:06:08.329788 8881 leader_election.cc:288] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa426e29ae2342c09714cbdeba8db23f (127.11.192.195:46369), 2fd1bd31c2b44db8af74961163f0f889 (127.11.192.193:41097)
I20260812 07:06:08.336550 8865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:08.330263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:51527 (local address 127.11.192.193:41097)
0812 07:06:08.330524 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:08.330531 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:08.331232 (+ 701us) server_negotiation.cc:407] Connection header received
0812 07:06:08.331412 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:08.331416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:08.331475 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:08.331548 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:06:08.332179 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.332868 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.334854 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.335635 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.335776 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:08.335848 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:08.336201 (+ 353us) server_negotiation.cc:299] Negotiation successful
0812 07:06:08.336329 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260812 07:06:08.336549 8863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:08.330236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:52945 (local address 127.11.192.195:46369)
0812 07:06:08.330445 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:08.330453 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:08.331201 (+ 748us) server_negotiation.cc:407] Connection header received
0812 07:06:08.331415 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:08.331419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:08.331478 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:08.331561 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:06:08.332220 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.332949 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.334846 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.335614 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.335756 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:08.335833 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:08.336177 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 07:06:08.336305 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 07:06:08.336807 8884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:08.330182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:46369 (local address 127.11.192.194:52945)
0812 07:06:08.331094 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:08.331112 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:06:08.331279 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:06:08.331587 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:06:08.331591 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:06:08.331623 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:06:08.332064 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:08.332096 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.333090 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.333095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:08.334694 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:08.334706 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.335847 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.335857 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:08.336007 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:08.336040 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:06:08.336145 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 07:06:08.336275 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"mutex_wait_us":52,"spinlock_wait_cycles":41088,"thread_start_us":450,"threads_started":1}
I20260812 07:06:08.336843 8888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:08.330284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:41097 (local address 127.11.192.194:51527)
0812 07:06:08.331113 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:08.331132 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:06:08.331280 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:06:08.331585 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:06:08.331590 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:06:08.331623 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:06:08.332007 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:08.332017 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.332991 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.332997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:08.334708 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:08.334721 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.335874 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.335882 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:08.336040 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:08.336064 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 07:06:08.336168 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 07:06:08.336297 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":106,"spinlock_wait_cycles":10624,"thread_start_us":435,"threads_started":1}
I20260812 07:06:08.337690 8761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db6c323bbb1e4af099d3cd54d8499432" candidate_uuid: "149bf9df73814bbabf72c34058df1ed1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa426e29ae2342c09714cbdeba8db23f" is_pre_election: true
I20260812 07:06:08.337697 8258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db6c323bbb1e4af099d3cd54d8499432" candidate_uuid: "149bf9df73814bbabf72c34058df1ed1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd1bd31c2b44db8af74961163f0f889" is_pre_election: true
I20260812 07:06:08.338140 8761 raft_consensus.cc:2454] T db6c323bbb1e4af099d3cd54d8499432 P fa426e29ae2342c09714cbdeba8db23f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 149bf9df73814bbabf72c34058df1ed1 in term 0.
I20260812 07:06:08.338151 8258 raft_consensus.cc:2454] T db6c323bbb1e4af099d3cd54d8499432 P 2fd1bd31c2b44db8af74961163f0f889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 149bf9df73814bbabf72c34058df1ed1 in term 0.
I20260812 07:06:08.338757 8377 leader_election.cc:302] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 149bf9df73814bbabf72c34058df1ed1, 2fd1bd31c2b44db8af74961163f0f889, fa426e29ae2342c09714cbdeba8db23f; no voters:
I20260812 07:06:08.339110 8881 raft_consensus.cc:2781] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:06:08.339242 8881 raft_consensus.cc:482] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:06:08.339332 8881 raft_consensus.cc:3037] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:06:08.341503 8881 raft_consensus.cc:504] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } }
I20260812 07:06:08.342017 8881 leader_election.cc:288] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1 [CANDIDATE]: Term 1 election: Requested vote from peers fa426e29ae2342c09714cbdeba8db23f (127.11.192.195:46369), 2fd1bd31c2b44db8af74961163f0f889 (127.11.192.193:41097)
I20260812 07:06:08.342504 8761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db6c323bbb1e4af099d3cd54d8499432" candidate_uuid: "149bf9df73814bbabf72c34058df1ed1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa426e29ae2342c09714cbdeba8db23f"
I20260812 07:06:08.342504 8258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db6c323bbb1e4af099d3cd54d8499432" candidate_uuid: "149bf9df73814bbabf72c34058df1ed1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd1bd31c2b44db8af74961163f0f889"
I20260812 07:06:08.342705 8761 raft_consensus.cc:3037] T db6c323bbb1e4af099d3cd54d8499432 P fa426e29ae2342c09714cbdeba8db23f [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:06:08.342705 8258 raft_consensus.cc:3037] T db6c323bbb1e4af099d3cd54d8499432 P 2fd1bd31c2b44db8af74961163f0f889 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:06:08.345829 8258 raft_consensus.cc:2454] T db6c323bbb1e4af099d3cd54d8499432 P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 149bf9df73814bbabf72c34058df1ed1 in term 1.
I20260812 07:06:08.345827 8761 raft_consensus.cc:2454] T db6c323bbb1e4af099d3cd54d8499432 P fa426e29ae2342c09714cbdeba8db23f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 149bf9df73814bbabf72c34058df1ed1 in term 1.
I20260812 07:06:08.346216 8377 leader_election.cc:302] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 149bf9df73814bbabf72c34058df1ed1, 2fd1bd31c2b44db8af74961163f0f889, fa426e29ae2342c09714cbdeba8db23f; no voters:
I20260812 07:06:08.346477 8881 raft_consensus.cc:2781] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:06:08.346956 8881 raft_consensus.cc:686] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1 [term 1 LEADER]: Becoming Leader. State: Replica: 149bf9df73814bbabf72c34058df1ed1, State: Running, Role: LEADER
I20260812 07:06:08.347404 8881 consensus_queue.cc:237] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } }
I20260812 07:06:08.351832 8003 catalog_manager.cc:5495] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 149bf9df73814bbabf72c34058df1ed1 (127.11.192.194). New cstate: current_term: 1 leader_uuid: "149bf9df73814bbabf72c34058df1ed1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } health_report { overall_health: HEALTHY } } }
W20260812 07:06:08.356115 8841 tablet.cc:2369] T db6c323bbb1e4af099d3cd54d8499432 P fa426e29ae2342c09714cbdeba8db23f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:06:08.361545 8329 tablet.cc:2369] T db6c323bbb1e4af099d3cd54d8499432 P 2fd1bd31c2b44db8af74961163f0f889: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:06:08.372036 8596 tablet.cc:2369] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:06:08.376451 8867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:08.371232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46440 (local address 127.11.192.194:38967)
0812 07:06:08.371408 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:08.371414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:08.371545 (+ 131us) server_negotiation.cc:407] Connection header received
0812 07:06:08.371741 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:08.371745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:08.371816 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:08.371938 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:06:08.372376 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.373087 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.374435 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.374837 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.374944 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:08.375429 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:06:08.375862 (+ 433us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:06:08.376008 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:08.376123 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 07:06:08.376205 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 07:06:08.383522 8258 raft_consensus.cc:1261] T db6c323bbb1e4af099d3cd54d8499432 P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Refusing update from remote peer 149bf9df73814bbabf72c34058df1ed1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:06:08.383562 8761 raft_consensus.cc:1261] T db6c323bbb1e4af099d3cd54d8499432 P fa426e29ae2342c09714cbdeba8db23f [term 1 FOLLOWER]: Refusing update from remote peer 149bf9df73814bbabf72c34058df1ed1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:06:08.384420 8889 consensus_queue.cc:1035] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:06:08.384764 8881 consensus_queue.cc:1035] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:06:08.395421 8897 mvcc.cc:205] Tried to move back new op lower bound from 7317579236890324992 to 7317579236754448384. Current Snapshot: MvccSnapshot[applied={T|T < 7317579236890324992}]
I20260812 07:06:08.408099 8003 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:06:08.408330 8003 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:06:08.411569 8003 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4: Sending DeleteTablet for 3 replicas of tablet db6c323bbb1e4af099d3cd54d8499432
I20260812 07:06:08.412596 8716 tablet_service.cc:1514] Processing DeleteTablet for tablet db6c323bbb1e4af099d3cd54d8499432 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:06:08 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:35584
I20260812 07:06:08.412966 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 8108
I20260812 07:06:08.413046 8461 tablet_service.cc:1514] Processing DeleteTablet for tablet db6c323bbb1e4af099d3cd54d8499432 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:06:08 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:46430
I20260812 07:06:08.414007 8925 tablet_replica.cc:330] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1: stopping tablet replica
I20260812 07:06:08.414378 8925 raft_consensus.cc:2229] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:06:08.414964 8925 raft_consensus.cc:2258] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:06:08.414939 8213 tablet_service.cc:1514] Processing DeleteTablet for tablet db6c323bbb1e4af099d3cd54d8499432 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:06:08 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:40024
I20260812 07:06:08.417033 8925 ts_tablet_manager.cc:1905] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:06:08.424307 8925 ts_tablet_manager.cc:1918] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:06:08.424535 8925 log.cc:1196] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/db6c323bbb1e4af099d3cd54d8499432
I20260812 07:06:08.425156 8925 ts_tablet_manager.cc:1939] T db6c323bbb1e4af099d3cd54d8499432 P 149bf9df73814bbabf72c34058df1ed1: Deleting consensus metadata
I20260812 07:06:08.426823 7981 catalog_manager.cc:4841] TS 149bf9df73814bbabf72c34058df1ed1 (127.11.192.194:38967): tablet db6c323bbb1e4af099d3cd54d8499432 (table test-table [id=0893bf56af504868b0f128c22080e266]) successfully deleted
I20260812 07:06:08.427649 8931 tablet_replica.cc:330] T db6c323bbb1e4af099d3cd54d8499432 P 2fd1bd31c2b44db8af74961163f0f889: stopping tablet replica
I20260812 07:06:08.428507 8923 tablet_replica.cc:330] T db6c323bbb1e4af099d3cd54d8499432 P fa426e29ae2342c09714cbdeba8db23f: stopping tablet replica
I20260812 07:06:08.428671 8931 raft_consensus.cc:2229] T db6c323bbb1e4af099d3cd54d8499432 P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:06:08.428807 8923 raft_consensus.cc:2229] T db6c323bbb1e4af099d3cd54d8499432 P fa426e29ae2342c09714cbdeba8db23f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:06:08.429142 8931 raft_consensus.cc:2258] T db6c323bbb1e4af099d3cd54d8499432 P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:06:08.429176 8923 raft_consensus.cc:2258] T db6c323bbb1e4af099d3cd54d8499432 P fa426e29ae2342c09714cbdeba8db23f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:06:08.430853 8923 ts_tablet_manager.cc:1905] T db6c323bbb1e4af099d3cd54d8499432 P fa426e29ae2342c09714cbdeba8db23f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:06:08.433166 8931 ts_tablet_manager.cc:1905] T db6c323bbb1e4af099d3cd54d8499432 P 2fd1bd31c2b44db8af74961163f0f889: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:06:08.436980 8923 ts_tablet_manager.cc:1918] T db6c323bbb1e4af099d3cd54d8499432 P fa426e29ae2342c09714cbdeba8db23f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:06:08.437237 8923 log.cc:1196] T db6c323bbb1e4af099d3cd54d8499432 P fa426e29ae2342c09714cbdeba8db23f: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/db6c323bbb1e4af099d3cd54d8499432
I20260812 07:06:08.437788 8923 ts_tablet_manager.cc:1939] T db6c323bbb1e4af099d3cd54d8499432 P fa426e29ae2342c09714cbdeba8db23f: Deleting consensus metadata
I20260812 07:06:08.439453 7983 catalog_manager.cc:4841] TS fa426e29ae2342c09714cbdeba8db23f (127.11.192.195:46369): tablet db6c323bbb1e4af099d3cd54d8499432 (table test-table [id=0893bf56af504868b0f128c22080e266]) successfully deleted
I20260812 07:06:08.439934 8931 ts_tablet_manager.cc:1918] T db6c323bbb1e4af099d3cd54d8499432 P 2fd1bd31c2b44db8af74961163f0f889: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:06:08.440124 8931 log.cc:1196] T db6c323bbb1e4af099d3cd54d8499432 P 2fd1bd31c2b44db8af74961163f0f889: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/wal/wals/db6c323bbb1e4af099d3cd54d8499432
Aug 12 07:06:08 dist-test-slave-lgr5 krb5kdc[7935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.193@KRBTEST.COM
I20260812 07:06:08.440632 8931 ts_tablet_manager.cc:1939] T db6c323bbb1e4af099d3cd54d8499432 P 2fd1bd31c2b44db8af74961163f0f889: Deleting consensus metadata
I20260812 07:06:08.442237 7981 catalog_manager.cc:4841] TS 2fd1bd31c2b44db8af74961163f0f889 (127.11.192.193:41097): tablet db6c323bbb1e4af099d3cd54d8499432 (table test-table [id=0893bf56af504868b0f128c22080e266]) successfully deleted
I20260812 07:06:08.446799 8865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:08.414346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40034 (local address 127.11.192.193:41097)
0812 07:06:08.414523 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:08.414530 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:08.428305 (+ 13775us) server_negotiation.cc:407] Connection header received
0812 07:06:08.429759 (+ 1454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:08.429765 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:08.429842 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:08.429932 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:06:08.431967 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.433069 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.435068 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.435711 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.435802 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:08.441213 (+ 5411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:06:08.441225 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:06:08.441238 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:06:08.441272 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:06:08.444116 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:08.444776 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:08.444785 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:08.444789 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:08.444920 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:08.445265 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:08.445271 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:08.445274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:08.445617 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:06:08.445893 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:08.446244 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 07:06:08.446446 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:06:08.447399 8319 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:06:08.597409 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 8108
I20260812 07:06:08.623797 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 8346
Aug 12 07:06:08 dist-test-slave-lgr5 krb5kdc[7935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.194@KRBTEST.COM
I20260812 07:06:08.643796 8867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:08.624607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46454 (local address 127.11.192.194:38967)
0812 07:06:08.624780 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:08.624786 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:08.624960 (+ 174us) server_negotiation.cc:407] Connection header received
0812 07:06:08.625139 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:08.625145 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:08.625233 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:08.625337 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:06:08.627031 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.628306 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.630236 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.630905 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.631042 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:08.636321 (+ 5279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:06:08.636337 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:06:08.636358 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:06:08.636409 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:06:08.640370 (+ 3961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:08.641169 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:08.641178 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:08.641182 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:08.641350 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:08.641753 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:08.641759 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:08.641763 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:08.642209 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:06:08.642531 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:08.642893 (+ 362us) server_negotiation.cc:299] Negotiation successful
0812 07:06:08.643426 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:06:08.644551 8584 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:06:08.758229 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 8346
I20260812 07:06:08.785343 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 8601
Aug 12 07:06:08 dist-test-slave-lgr5 krb5kdc[7935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.195@KRBTEST.COM
I20260812 07:06:08.799829 8863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:08.786094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35600 (local address 127.11.192.195:46369)
0812 07:06:08.786263 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:08.786267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:08.786337 (+ 70us) server_negotiation.cc:407] Connection header received
0812 07:06:08.786528 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:08.786532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:08.786604 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:08.786680 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:06:08.788276 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.788965 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.790173 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:08.790535 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:08.790624 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:08.795067 (+ 4443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:06:08.795082 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:06:08.795098 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:06:08.795130 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:06:08.797564 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:08.798163 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:08.798169 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:08.798172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:08.798296 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:08.798617 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:08.798621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:08.798623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:08.798942 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:06:08.799165 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:08.799376 (+ 211us) server_negotiation.cc:299] Negotiation successful
0812 07:06:08.799526 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:06:08.800462 8833 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:06:08.886536 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 8601
I20260812 07:06:08.912443 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 7961
I20260812 07:06:08.913518 8051 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:06:09.020732 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 7961
I20260812 07:06:29.051539 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:37881
--webserver_interface=127.11.192.254
--webserver_port=46593
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:37881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:06:29.378748 8938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:06:29.379006 8938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:06:29.379102 8938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:06:29.385989 8938 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 07:06:29.386073 8938 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 07:06:29.386118 8938 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 07:06:29.386168 8938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:06:29.386224 8938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:06:29.386289 8938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:06:29.386332 8938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:06:29.386384 8938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:06:29.395702 8938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:37881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:37881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=46593
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:06:29.396840 8938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:06:29.398113 8938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:06:29.421012 8946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:29.420989 8947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:29.421140 8950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:06:29.422123 8938 server_base.cc:1034] running on GCE node
I20260812 07:06:29.422899 8938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:06:29.424397 8938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:06:29.425622 8938 hybrid_clock.cc:648] HybridClock initialized: now 1786518389425559 us; error 30 us; skew 500 ppm
I20260812 07:06:29.425963 8938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:06:29 dist-test-slave-lgr5 krb5kdc[7935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518389, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:06:29.429386 8938 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:06:29.430867 8938 webserver.cc:460] Webserver started at http://127.11.192.254:46593/ using document root <none> and password file <none>
I20260812 07:06:29.431599 8938 fs_manager.cc:359] Metadata directory not provided
I20260812 07:06:29.431715 8938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:06:29.437608 8938 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260812 07:06:29.440524 8965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:29.441854 8938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:06:29.442003 8938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "4ee74e07b85b420d9a3faea5221e66a4"
format_stamp: "Formatted at 2026-08-12 07:06:06 on dist-test-slave-lgr5"
server_key: "f10d94cc9bbf70796bd69cd8d36a8bff"
server_key_iv: "a485227446a83ca1386e99541ff35f0c"
server_key_version: "encryptionkey@0"
I20260812 07:06:29.443002 8938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:06:29.459791 8938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:06:29.463544 8938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:06:29.463773 8938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:06:29.481262 8938 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:37881
I20260812 07:06:29.481292 9051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:37881 every 8 connection(s)
I20260812 07:06:29.482707 8938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:06:29.487383 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 8938
I20260812 07:06:29.488234 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:41097
--local_ip_for_outbound_sockets=127.11.192.193
--tserver_master_addrs=127.11.192.254:37881
--webserver_port=41117
--webserver_interface=127.11.192.193
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:06:29.489811 9053 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:06:29.493011 9053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4: Bootstrap starting.
I20260812 07:06:29.515403 9053 log.cc:824] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4: Log is configured to *not* fsync() on all Append() calls
I20260812 07:06:29.530946 9053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:06:29.531574 9053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4: Bootstrap complete.
I20260812 07:06:29.544762 9053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee74e07b85b420d9a3faea5221e66a4" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 37881 } }
I20260812 07:06:29.545879 9053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee74e07b85b420d9a3faea5221e66a4, State: Initialized, Role: FOLLOWER
I20260812 07:06:29.546630 9053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee74e07b85b420d9a3faea5221e66a4" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 37881 } }
I20260812 07:06:29.546903 9053 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:06:29.547024 9053 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:06:29.547192 9053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 1 FOLLOWER]: Advancing to term 2
I20260812 07:06:29.552743 9053 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee74e07b85b420d9a3faea5221e66a4" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 37881 } }
I20260812 07:06:29.553388 9053 leader_election.cc:302] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ee74e07b85b420d9a3faea5221e66a4; no voters:
I20260812 07:06:29.554037 9053 leader_election.cc:288] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 07:06:29.554184 9063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 07:06:29.555454 9063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 2 LEADER]: Becoming Leader. State: Replica: 4ee74e07b85b420d9a3faea5221e66a4, State: Running, Role: LEADER
I20260812 07:06:29.556030 9063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee74e07b85b420d9a3faea5221e66a4" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 37881 } }
I20260812 07:06:29.556303 9053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:06:29.558653 9064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4ee74e07b85b420d9a3faea5221e66a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee74e07b85b420d9a3faea5221e66a4" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 37881 } } }
I20260812 07:06:29.559016 9064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [sys.catalog]: This master's current role is: LEADER
I20260812 07:06:29.558617 9065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ee74e07b85b420d9a3faea5221e66a4. Latest consensus state: current_term: 2 leader_uuid: "4ee74e07b85b420d9a3faea5221e66a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee74e07b85b420d9a3faea5221e66a4" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 37881 } } }
I20260812 07:06:29.559933 9065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [sys.catalog]: This master's current role is: LEADER
I20260812 07:06:29.561620 9078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:06:29.569806 9078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:06:29.571303 9078 catalog_manager.cc:1259] Loaded cluster ID: 8f2470c482394ec7b633fbb5816820f0
I20260812 07:06:29.571460 9078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:06:29.575691 9078 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:06:29.577358 9078 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4: Loaded TSK: 0
I20260812 07:06:29.595788 9099 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 07:06:29.605706 9078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4: Generated new TSK 1
I20260812 07:06:29.606112 9078 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 07:06:29.844733 9058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:06:29.844996 9058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:06:29.845132 9058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:06:29.852222 9058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:06:29.852330 9058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:06:29.852474 9058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:06:29.862509 9058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:41097
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=41117
--tserver_master_addrs=127.11.192.254:37881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:06:29.863725 9058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:06:29.865028 9058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:06:29.879263 9110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:29.879704 9113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:29.879338 9109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:06:29.880160 9058 server_base.cc:1034] running on GCE node
I20260812 07:06:29.880726 9058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:06:29.881644 9058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:06:29.882855 9058 hybrid_clock.cc:648] HybridClock initialized: now 1786518389882813 us; error 36 us; skew 500 ppm
I20260812 07:06:29.883142 9058 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:06:29 dist-test-slave-lgr5 krb5kdc[7935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518389, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:06:29.886281 9058 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:06:29.887673 9058 webserver.cc:460] Webserver started at http://127.11.192.193:41117/ using document root <none> and password file <none>
I20260812 07:06:29.888327 9058 fs_manager.cc:359] Metadata directory not provided
I20260812 07:06:29.888458 9058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:06:29.893759 9058 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260812 07:06:29.896477 9128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:29.897840 9058 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260812 07:06:29.897995 9058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "2fd1bd31c2b44db8af74961163f0f889"
format_stamp: "Formatted at 2026-08-12 07:06:07 on dist-test-slave-lgr5"
server_key: "6c62a0211db307d9bc49b8b898fb0f9c"
server_key_iv: "a528c1d510c621f597de99da4b75fd6f"
server_key_version: "encryptionkey@0"
I20260812 07:06:29.898916 9058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:06:29.914196 9058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:06:29.918229 9058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:06:29.918465 9058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:06:29.919411 9058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:06:29.920820 9058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:06:29.920917 9058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:29.921018 9058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:06:29.921092 9058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:29.953506 9058 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:41097
I20260812 07:06:29.953576 9333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:41097 every 8 connection(s)
I20260812 07:06:29.954823 9058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:06:29.961301 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 9058
I20260812 07:06:29.962126 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:38967
--local_ip_for_outbound_sockets=127.11.192.194
--tserver_master_addrs=127.11.192.254:37881
--webserver_port=46083
--webserver_interface=127.11.192.194
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:06:29 dist-test-slave-lgr5 krb5kdc[7935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518389, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:06:29.981151 9338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:29.957787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.193:57567)
0812 07:06:29.958616 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:29.958678 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 07:06:29.960520 (+ 1842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:06:29.961802 (+ 1282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:06:29.961814 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:06:29.962534 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:06:29.963281 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:29.963322 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:29.964970 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:29.964977 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:29.966767 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:29.966786 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:29.967922 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:29.967930 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:29.968250 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:29.968953 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:06:29.968985 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:06:29.971628 (+ 2643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:06:29.976215 (+ 4587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:06:29.976223 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:06:29.976234 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:06:29.976856 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:06:29.977400 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:06:29.977406 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:06:29.977410 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:06:29.977601 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:06:29.978445 (+ 844us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:06:29.978453 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:06:29.978912 (+ 459us) client_negotiation.cc:769] Sending connection context
0812 07:06:29.979228 (+ 316us) client_negotiation.cc:240] Negotiation successful
0812 07:06:29.979722 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":330,"threads_started":1}
I20260812 07:06:29.981593 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:29.957593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:57567 (local address 127.11.192.254:37881)
0812 07:06:29.958137 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:29.958148 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:29.958792 (+ 644us) server_negotiation.cc:407] Connection header received
0812 07:06:29.960971 (+ 2179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:29.960983 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:29.961276 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:29.961585 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:06:29.963507 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:29.964801 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:29.966935 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:29.967683 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:29.967812 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:29.971980 (+ 4168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:06:29.971992 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:06:29.972010 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:06:29.972057 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:06:29.976002 (+ 3945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:29.977003 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:29.977028 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:29.977036 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:29.977257 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:29.977713 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:29.977719 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:29.977722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:29.978288 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:06:29.978497 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:29.979654 (+ 1157us) server_negotiation.cc:299] Negotiation successful
0812 07:06:29.980290 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":235,"threads_started":1}
I20260812 07:06:29.984545 9334 heartbeater.cc:347] Connected to a master server at 127.11.192.254:37881
I20260812 07:06:29.984815 9334 heartbeater.cc:464] Registering TS with master...
I20260812 07:06:29.985499 9334 heartbeater.cc:511] Master 127.11.192.254:37881 requested a full tablet report, sending...
I20260812 07:06:29.987994 8996 ts_manager.cc:194] Registered new tserver with Master: 2fd1bd31c2b44db8af74961163f0f889 (127.11.192.193:41097)
I20260812 07:06:29.990156 8996 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:57567
W20260812 07:06:30.294821 9339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:06:30.295086 9339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:06:30.295187 9339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:06:30.302112 9339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:06:30.302202 9339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:06:30.302317 9339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:06:30.311517 9339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:38967
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=46083
--tserver_master_addrs=127.11.192.254:37881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:06:30.312639 9339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:06:30.313920 9339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:06:30.325911 9351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:30.325953 9352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:30.325986 9354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:06:30.326721 9339 server_base.cc:1034] running on GCE node
I20260812 07:06:30.327253 9339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:06:30.328140 9339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:06:30.329357 9339 hybrid_clock.cc:648] HybridClock initialized: now 1786518390329301 us; error 42 us; skew 500 ppm
I20260812 07:06:30.329663 9339 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:06:30 dist-test-slave-lgr5 krb5kdc[7935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518390, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:06:30.332599 9339 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:06:30.333905 9339 webserver.cc:460] Webserver started at http://127.11.192.194:46083/ using document root <none> and password file <none>
I20260812 07:06:30.334530 9339 fs_manager.cc:359] Metadata directory not provided
I20260812 07:06:30.334657 9339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:06:30.339677 9339 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:06:30.342289 9371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:30.343523 9339 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 07:06:30.343675 9339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "149bf9df73814bbabf72c34058df1ed1"
format_stamp: "Formatted at 2026-08-12 07:06:07 on dist-test-slave-lgr5"
server_key: "71c6818c8664e1c9f1d62c15c87dabc4"
server_key_iv: "b40201d1784c17dc0e4c8f7f140bcb6a"
server_key_version: "encryptionkey@0"
I20260812 07:06:30.344673 9339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:06:30.366925 9339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:06:30.370770 9339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:06:30.371021 9339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:06:30.371867 9339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:06:30.373123 9339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:06:30.373221 9339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:30.373324 9339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:06:30.373396 9339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:30.402587 9339 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:38967
I20260812 07:06:30.402659 9556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:38967 every 8 connection(s)
I20260812 07:06:30.403997 9339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:06:30.404465 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 9339
I20260812 07:06:30.405233 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:46369
--local_ip_for_outbound_sockets=127.11.192.195
--tserver_master_addrs=127.11.192.254:37881
--webserver_port=36431
--webserver_interface=127.11.192.195
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:06:30 dist-test-slave-lgr5 krb5kdc[7935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518390, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:06:30.423979 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:30.406615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:33121 (local address 127.11.192.254:37881)
0812 07:06:30.406817 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:30.406824 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:30.407929 (+ 1105us) server_negotiation.cc:407] Connection header received
0812 07:06:30.410172 (+ 2243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:30.410176 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:30.410242 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:30.410328 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:06:30.412163 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.412851 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.414849 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.415281 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.415376 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:30.418722 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:06:30.418727 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:06:30.418729 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:06:30.418761 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:06:30.421146 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:30.421817 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:30.421821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:30.421823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:30.421940 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:30.422343 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:30.422346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:30.422348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:30.422616 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:06:30.422749 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:30.423390 (+ 641us) server_negotiation.cc:299] Negotiation successful
0812 07:06:30.423631 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:06:30.424809 9559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:30.406990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.194:33121)
0812 07:06:30.407763 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:30.407808 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 07:06:30.409911 (+ 2103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:06:30.410575 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:06:30.410585 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:06:30.411234 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:06:30.411977 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:30.412004 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.413011 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.413016 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:30.414714 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:30.414730 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.415484 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.415492 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:30.415756 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:30.416432 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:06:30.416454 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:06:30.418503 (+ 2049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:06:30.421286 (+ 2783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:06:30.421295 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:06:30.421309 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:06:30.421693 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:06:30.422040 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:06:30.422045 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:06:30.422049 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:06:30.422229 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:06:30.422738 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:06:30.422745 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:06:30.423046 (+ 301us) client_negotiation.cc:769] Sending connection context
0812 07:06:30.423266 (+ 220us) client_negotiation.cc:240] Negotiation successful
0812 07:06:30.423728 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":293,"threads_started":1}
I20260812 07:06:30.426136 9557 heartbeater.cc:347] Connected to a master server at 127.11.192.254:37881
I20260812 07:06:30.426398 9557 heartbeater.cc:464] Registering TS with master...
I20260812 07:06:30.427048 9557 heartbeater.cc:511] Master 127.11.192.254:37881 requested a full tablet report, sending...
I20260812 07:06:30.428459 8996 ts_manager.cc:194] Registered new tserver with Master: 149bf9df73814bbabf72c34058df1ed1 (127.11.192.194:38967)
I20260812 07:06:30.429533 8996 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:33121
W20260812 07:06:30.746126 9562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:06:30.746413 9562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:06:30.746521 9562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:06:30.753603 9562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:06:30.753706 9562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:06:30.753826 9562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:06:30.763166 9562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:46369
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=36431
--tserver_master_addrs=127.11.192.254:37881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:06:30.764267 9562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:06:30.765480 9562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:06:30.777729 9571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:30.777758 9574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:30.777792 9572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:06:30.778586 9562 server_base.cc:1034] running on GCE node
I20260812 07:06:30.779125 9562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:06:30.779953 9562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:06:30.781153 9562 hybrid_clock.cc:648] HybridClock initialized: now 1786518390781113 us; error 48 us; skew 500 ppm
I20260812 07:06:30.781452 9562 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:06:30 dist-test-slave-lgr5 krb5kdc[7935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518390, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:06:30.784562 9562 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:06:30.785953 9562 webserver.cc:460] Webserver started at http://127.11.192.195:36431/ using document root <none> and password file <none>
I20260812 07:06:30.786612 9562 fs_manager.cc:359] Metadata directory not provided
I20260812 07:06:30.786749 9562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:06:30.791960 9562 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:06:30.794570 9586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:30.795804 9562 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:06:30.795961 9562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "fa426e29ae2342c09714cbdeba8db23f"
format_stamp: "Formatted at 2026-08-12 07:06:08 on dist-test-slave-lgr5"
server_key: "9fb6cdf0a648a4ff9b3c03c008a135e3"
server_key_iv: "ecabdf9367798e0481d0a46251b5b0e7"
server_key_version: "encryptionkey@0"
I20260812 07:06:30.796955 9562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:06:30.813752 9562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:06:30.817441 9562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:06:30.817721 9562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:06:30.818712 9562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:06:30.819960 9562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:06:30.820063 9562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:30.820189 9562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:06:30.820256 9562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:30.854097 9562 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:46369
I20260812 07:06:30.854147 9785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:46369 every 8 connection(s)
I20260812 07:06:30.855538 9562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:06:30.862809 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 9562
Aug 12 07:06:30 dist-test-slave-lgr5 krb5kdc[7935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518390, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:06:30.876052 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:30.857957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:48869 (local address 127.11.192.254:37881)
0812 07:06:30.858170 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:30.858177 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:30.859251 (+ 1074us) server_negotiation.cc:407] Connection header received
0812 07:06:30.861248 (+ 1997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:30.861253 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:30.861318 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:30.861400 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:06:30.862937 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.863887 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.865483 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.866088 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.866202 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:30.869726 (+ 3524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:06:30.869733 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:06:30.869738 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:06:30.869783 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:06:30.873219 (+ 3436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:30.873946 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:30.873952 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:30.873956 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:30.874122 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:30.874520 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:30.874525 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:30.874529 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:30.874884 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:06:30.875065 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:30.875604 (+ 539us) server_negotiation.cc:299] Negotiation successful
0812 07:06:30.875763 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":61}
I20260812 07:06:30.877017 9791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:30.858386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.195:48869)
0812 07:06:30.859082 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:30.859132 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 07:06:30.861006 (+ 1874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:06:30.861596 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:06:30.861604 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:06:30.862149 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:06:30.862738 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:30.862756 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.864159 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.864163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:30.865339 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:30.865352 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.866313 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.866320 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:30.866543 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:30.867169 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:06:30.867194 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:06:30.869526 (+ 2332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:06:30.873377 (+ 3851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:06:30.873386 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:06:30.873400 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:06:30.873806 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:06:30.874226 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:06:30.874229 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:06:30.874231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:06:30.874409 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:06:30.875006 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:06:30.875011 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:06:30.875287 (+ 276us) client_negotiation.cc:769] Sending connection context
0812 07:06:30.875495 (+ 208us) client_negotiation.cc:240] Negotiation successful
0812 07:06:30.875821 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":262,"threads_started":1}
I20260812 07:06:30.878495 9789 heartbeater.cc:347] Connected to a master server at 127.11.192.254:37881
I20260812 07:06:30.878782 9789 heartbeater.cc:464] Registering TS with master...
I20260812 07:06:30.879483 9789 heartbeater.cc:511] Master 127.11.192.254:37881 requested a full tablet report, sending...
I20260812 07:06:30.881031 8996 ts_manager.cc:194] Registered new tserver with Master: fa426e29ae2342c09714cbdeba8db23f (127.11.192.195:46369)
I20260812 07:06:30.882093 8996 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:48869
I20260812 07:06:30.882153 9792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:30.863769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45302 (local address 127.11.192.254:37881)
0812 07:06:30.864335 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:30.864343 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:30.865877 (+ 1534us) server_negotiation.cc:407] Connection header received
0812 07:06:30.866035 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:30.866040 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:30.866098 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:30.866190 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:06:30.868292 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.869150 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.871151 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.871676 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.871764 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:30.876469 (+ 4705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:06:30.876487 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:06:30.876491 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:06:30.876525 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:06:30.879384 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:30.880025 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:30.880030 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:30.880032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:30.880174 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:30.880648 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:30.880652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:30.880655 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:30.880981 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:06:30.881256 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:30.881560 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 07:06:30.881775 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":408,"thread_start_us":232,"threads_started":1}
I20260812 07:06:30.884537 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:06:30.890134 9792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:30.885385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45310 (local address 127.11.192.254:37881)
0812 07:06:30.885543 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:30.885547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:30.885931 (+ 384us) server_negotiation.cc:407] Connection header received
0812 07:06:30.886081 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:30.886084 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:30.886193 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:30.886298 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:06:30.886622 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.887266 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.888647 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.889071 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.889156 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:30.889545 (+ 389us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:06:30.889720 (+ 175us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 07:06:30.889944 (+ 224us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45310: token expired
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 07:06:30.890216 9792 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45310: token expired
W20260812 07:06:30.890738 8845 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.11.192.254:37881)
I20260812 07:06:30.901854 9792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:30.891209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45318 (local address 127.11.192.254:37881)
0812 07:06:30.891338 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:30.891342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:30.891402 (+ 60us) server_negotiation.cc:407] Connection header received
0812 07:06:30.891516 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:30.891520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:30.891565 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:30.891629 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:06:30.892861 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.893508 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.894728 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.895113 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.895193 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:30.897430 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:06:30.897434 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:06:30.897437 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:06:30.897463 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:06:30.899552 (+ 2089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:30.900206 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:30.900209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:30.900211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:30.900340 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:30.900635 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:30.900638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:30.900640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:30.900911 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:06:30.901104 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:30.901382 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 07:06:30.901562 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 07:06:30.904860 8996 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:06:30.911717 8996 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:06:30.935130 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:30.923175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50222 (local address 127.11.192.194:38967)
0812 07:06:30.924062 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:30.924071 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:30.924780 (+ 709us) server_negotiation.cc:407] Connection header received
0812 07:06:30.924894 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:30.924902 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:30.925148 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:30.925338 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:06:30.926327 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.928204 (+ 1877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.932235 (+ 4031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.933645 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.933786 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:30.934587 (+ 801us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:30.934660 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 07:06:30.934818 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":556,"threads_started":1}
I20260812 07:06:30.935384 9810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:30.922539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:38967 (local address 127.0.0.1:50222)
0812 07:06:30.924519 (+ 1980us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:30.924537 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:06:30.924658 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:06:30.925687 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:06:30.925695 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:06:30.925726 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:06:30.926106 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:30.926117 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.928370 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.928376 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:30.932035 (+ 3659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:30.932046 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.933984 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.933990 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:30.934213 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:30.934320 (+ 107us) client_negotiation.cc:769] Sending connection context
0812 07:06:30.934455 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 07:06:30.934816 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1852,"mutex_wait_us":160,"thread_start_us":1728,"threads_started":1}
I20260812 07:06:30.938827 9815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:30.924624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:41097 (local address 127.0.0.1:44784)
0812 07:06:30.931222 (+ 6598us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:30.931237 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:06:30.931337 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:06:30.932013 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:06:30.932021 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:06:30.932044 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:06:30.932361 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:30.932370 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.933620 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.933625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:30.935984 (+ 2359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:30.936001 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.937846 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.937853 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:30.938016 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:30.938048 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 07:06:30.938140 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 07:06:30.938254 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6495,"mutex_wait_us":185,"thread_start_us":6377,"threads_started":1}
I20260812 07:06:30.939077 9817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:30.924870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44784 (local address 127.11.192.193:41097)
0812 07:06:30.931582 (+ 6712us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:30.931587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:30.931608 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:06:30.931683 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:30.931688 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:30.931823 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:30.931989 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:06:30.932517 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.933496 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.936172 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.937620 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.937785 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:30.937943 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:30.938415 (+ 472us) server_negotiation.cc:299] Negotiation successful
0812 07:06:30.938576 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6561,"thread_start_us":6427,"threads_started":1}
I20260812 07:06:30.939450 9806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:30.922404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:46369 (local address 127.0.0.1:33442)
0812 07:06:30.923209 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:30.923257 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 07:06:30.923429 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:06:30.931523 (+ 8094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:06:30.931528 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:06:30.931552 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:06:30.931908 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:30.931923 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.934206 (+ 2283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.934211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:30.936431 (+ 2220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:30.936446 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.938145 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.938151 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:30.938310 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:30.938341 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 07:06:30.938426 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 07:06:30.938561 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"mutex_wait_us":2,"thread_start_us":413,"threads_started":1}
I20260812 07:06:30.938944 9447 tablet_service.cc:1467] Processing CreateTablet for tablet 92c0ea9d5fb743efb9108415b984c8dc (DEFAULT_TABLE table=test-table [id=626a525f7fb643bcbc13e9bd71629c5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:06:30.940147 9811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:30.922510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33442 (local address 127.11.192.195:46369)
0812 07:06:30.930973 (+ 8463us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:30.930980 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:30.931024 (+ 44us) server_negotiation.cc:407] Connection header received
0812 07:06:30.931113 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:30.931121 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:30.931334 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:30.932147 (+ 813us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:06:30.932403 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.934051 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.936621 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:30.937934 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:30.938122 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:30.938281 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:30.939667 (+ 1386us) server_negotiation.cc:299] Negotiation successful
0812 07:06:30.939817 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8306,"thread_start_us":8145,"threads_started":1}
I20260812 07:06:30.940416 9447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92c0ea9d5fb743efb9108415b984c8dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:06:30.942587 9211 tablet_service.cc:1467] Processing CreateTablet for tablet 92c0ea9d5fb743efb9108415b984c8dc (DEFAULT_TABLE table=test-table [id=626a525f7fb643bcbc13e9bd71629c5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:06:30.943912 9211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92c0ea9d5fb743efb9108415b984c8dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:06:30.943647 9681 tablet_service.cc:1467] Processing CreateTablet for tablet 92c0ea9d5fb743efb9108415b984c8dc (DEFAULT_TABLE table=test-table [id=626a525f7fb643bcbc13e9bd71629c5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:06:30.945387 9681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92c0ea9d5fb743efb9108415b984c8dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:06:30.958832 9818 tablet_bootstrap.cc:492] T 92c0ea9d5fb743efb9108415b984c8dc P 149bf9df73814bbabf72c34058df1ed1: Bootstrap starting.
I20260812 07:06:30.964633 9821 tablet_bootstrap.cc:492] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889: Bootstrap starting.
I20260812 07:06:30.966398 9818 tablet_bootstrap.cc:654] T 92c0ea9d5fb743efb9108415b984c8dc P 149bf9df73814bbabf72c34058df1ed1: Neither blocks nor log segments found. Creating new log.
I20260812 07:06:30.967239 9822 tablet_bootstrap.cc:492] T 92c0ea9d5fb743efb9108415b984c8dc P fa426e29ae2342c09714cbdeba8db23f: Bootstrap starting.
I20260812 07:06:30.967515 9821 tablet_bootstrap.cc:654] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889: Neither blocks nor log segments found. Creating new log.
I20260812 07:06:30.967676 9818 log.cc:824] T 92c0ea9d5fb743efb9108415b984c8dc P 149bf9df73814bbabf72c34058df1ed1: Log is configured to *not* fsync() on all Append() calls
I20260812 07:06:30.968778 9821 log.cc:824] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889: Log is configured to *not* fsync() on all Append() calls
I20260812 07:06:30.970603 9818 tablet_bootstrap.cc:492] T 92c0ea9d5fb743efb9108415b984c8dc P 149bf9df73814bbabf72c34058df1ed1: No bootstrap required, opened a new log
I20260812 07:06:30.970659 9822 tablet_bootstrap.cc:654] T 92c0ea9d5fb743efb9108415b984c8dc P fa426e29ae2342c09714cbdeba8db23f: Neither blocks nor log segments found. Creating new log.
I20260812 07:06:30.970961 9818 ts_tablet_manager.cc:1397] T 92c0ea9d5fb743efb9108415b984c8dc P 149bf9df73814bbabf72c34058df1ed1: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.002s
I20260812 07:06:30.971777 9821 tablet_bootstrap.cc:492] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889: No bootstrap required, opened a new log
I20260812 07:06:30.972046 9821 ts_tablet_manager.cc:1397] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 07:06:30.972215 9822 log.cc:824] T 92c0ea9d5fb743efb9108415b984c8dc P fa426e29ae2342c09714cbdeba8db23f: Log is configured to *not* fsync() on all Append() calls
I20260812 07:06:30.975162 9822 tablet_bootstrap.cc:492] T 92c0ea9d5fb743efb9108415b984c8dc P fa426e29ae2342c09714cbdeba8db23f: No bootstrap required, opened a new log
I20260812 07:06:30.975406 9822 ts_tablet_manager.cc:1397] T 92c0ea9d5fb743efb9108415b984c8dc P fa426e29ae2342c09714cbdeba8db23f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 07:06:30.978330 9818 raft_consensus.cc:348] T 92c0ea9d5fb743efb9108415b984c8dc P 149bf9df73814bbabf72c34058df1ed1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } }
I20260812 07:06:30.978647 9818 raft_consensus.cc:374] T 92c0ea9d5fb743efb9108415b984c8dc P 149bf9df73814bbabf72c34058df1ed1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:06:30.978745 9818 raft_consensus.cc:729] T 92c0ea9d5fb743efb9108415b984c8dc P 149bf9df73814bbabf72c34058df1ed1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 149bf9df73814bbabf72c34058df1ed1, State: Initialized, Role: FOLLOWER
I20260812 07:06:30.979285 9818 consensus_queue.cc:260] T 92c0ea9d5fb743efb9108415b984c8dc P 149bf9df73814bbabf72c34058df1ed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } }
I20260812 07:06:30.980304 9557 heartbeater.cc:503] Master 127.11.192.254:37881 was elected leader, sending a full tablet report...
I20260812 07:06:30.980767 9818 ts_tablet_manager.cc:1428] T 92c0ea9d5fb743efb9108415b984c8dc P 149bf9df73814bbabf72c34058df1ed1: Time spent starting tablet: real 0.010s user 0.007s sys 0.002s
I20260812 07:06:30.983335 9821 raft_consensus.cc:348] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } }
I20260812 07:06:30.983772 9821 raft_consensus.cc:374] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:06:30.983879 9821 raft_consensus.cc:729] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd1bd31c2b44db8af74961163f0f889, State: Initialized, Role: FOLLOWER
I20260812 07:06:30.984598 9821 consensus_queue.cc:260] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } }
I20260812 07:06:30.985208 9822 raft_consensus.cc:348] T 92c0ea9d5fb743efb9108415b984c8dc P fa426e29ae2342c09714cbdeba8db23f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } }
I20260812 07:06:30.985647 9822 raft_consensus.cc:374] T 92c0ea9d5fb743efb9108415b984c8dc P fa426e29ae2342c09714cbdeba8db23f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:06:30.985702 9334 heartbeater.cc:503] Master 127.11.192.254:37881 was elected leader, sending a full tablet report...
I20260812 07:06:30.985788 9822 raft_consensus.cc:729] T 92c0ea9d5fb743efb9108415b984c8dc P fa426e29ae2342c09714cbdeba8db23f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa426e29ae2342c09714cbdeba8db23f, State: Initialized, Role: FOLLOWER
I20260812 07:06:30.986362 9821 ts_tablet_manager.cc:1428] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889: Time spent starting tablet: real 0.014s user 0.004s sys 0.008s
I20260812 07:06:30.986485 9822 consensus_queue.cc:260] T 92c0ea9d5fb743efb9108415b984c8dc P fa426e29ae2342c09714cbdeba8db23f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } }
I20260812 07:06:30.987517 9789 heartbeater.cc:503] Master 127.11.192.254:37881 was elected leader, sending a full tablet report...
I20260812 07:06:30.988000 9822 ts_tablet_manager.cc:1428] T 92c0ea9d5fb743efb9108415b984c8dc P fa426e29ae2342c09714cbdeba8db23f: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
W20260812 07:06:31.107131 9790 tablet.cc:2369] T 92c0ea9d5fb743efb9108415b984c8dc P fa426e29ae2342c09714cbdeba8db23f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:06:31.155788 9558 tablet.cc:2369] T 92c0ea9d5fb743efb9108415b984c8dc P 149bf9df73814bbabf72c34058df1ed1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:06:31.177084 9831 raft_consensus.cc:482] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:06:31.177372 9831 raft_consensus.cc:504] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } }
I20260812 07:06:31.179051 9831 leader_election.cc:288] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa426e29ae2342c09714cbdeba8db23f (127.11.192.195:46369), 149bf9df73814bbabf72c34058df1ed1 (127.11.192.194:38967)
I20260812 07:06:31.185534 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:31.179703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:50961 (local address 127.11.192.194:38967)
0812 07:06:31.179898 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:31.179904 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:31.180502 (+ 598us) server_negotiation.cc:407] Connection header received
0812 07:06:31.180673 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:31.180677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:31.180735 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:31.180808 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:06:31.181408 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:31.182069 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:31.183854 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:31.184649 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:31.184783 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:31.184859 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:31.185150 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 07:06:31.185275 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:06:31.185580 9811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:31.179355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:38045 (local address 127.11.192.195:46369)
0812 07:06:31.179564 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:31.179569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:31.180422 (+ 853us) server_negotiation.cc:407] Connection header received
0812 07:06:31.180601 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:31.180605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:31.180666 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:31.180739 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:06:31.181312 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:31.182025 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:31.183840 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:31.184663 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:31.184809 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:31.184885 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:31.185205 (+ 320us) server_negotiation.cc:299] Negotiation successful
0812 07:06:31.185330 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 07:06:31.185649 9834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:31.179544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:38967 (local address 127.11.192.193:50961)
0812 07:06:31.180402 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:31.180422 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:06:31.180551 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:06:31.180832 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:06:31.180836 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:06:31.180865 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:06:31.181256 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:31.181265 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:31.182187 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:31.182192 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:31.183710 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:31.183723 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:31.184862 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:31.184868 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:31.185004 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:31.185028 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 07:06:31.185108 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 07:06:31.185239 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"mutex_wait_us":30,"thread_start_us":440,"threads_started":1}
I20260812 07:06:31.185693 9833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:31.179427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:46369 (local address 127.11.192.193:38045)
0812 07:06:31.180309 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:31.180328 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:06:31.180450 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:06:31.180773 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:06:31.180778 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:06:31.180802 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:06:31.181167 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:31.181180 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:31.182162 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:31.182165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:31.183689 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:31.183702 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:31.184918 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:31.184924 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:31.185053 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:31.185076 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 07:06:31.185155 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 07:06:31.185281 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"mutex_wait_us":28,"thread_start_us":473,"threads_started":1}
I20260812 07:06:31.186643 9711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92c0ea9d5fb743efb9108415b984c8dc" candidate_uuid: "2fd1bd31c2b44db8af74961163f0f889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa426e29ae2342c09714cbdeba8db23f" is_pre_election: true
I20260812 07:06:31.187059 9485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92c0ea9d5fb743efb9108415b984c8dc" candidate_uuid: "2fd1bd31c2b44db8af74961163f0f889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "149bf9df73814bbabf72c34058df1ed1" is_pre_election: true
I20260812 07:06:31.187227 9711 raft_consensus.cc:2454] T 92c0ea9d5fb743efb9108415b984c8dc P fa426e29ae2342c09714cbdeba8db23f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fd1bd31c2b44db8af74961163f0f889 in term 0.
I20260812 07:06:31.187577 9485 raft_consensus.cc:2454] T 92c0ea9d5fb743efb9108415b984c8dc P 149bf9df73814bbabf72c34058df1ed1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fd1bd31c2b44db8af74961163f0f889 in term 0.
I20260812 07:06:31.187914 9133 leader_election.cc:302] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd1bd31c2b44db8af74961163f0f889, fa426e29ae2342c09714cbdeba8db23f; no voters:
I20260812 07:06:31.188340 9831 raft_consensus.cc:2781] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:06:31.188496 9831 raft_consensus.cc:482] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:06:31.188596 9831 raft_consensus.cc:3037] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:06:31.190650 9831 raft_consensus.cc:504] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } }
I20260812 07:06:31.191206 9831 leader_election.cc:288] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 1 election: Requested vote from peers fa426e29ae2342c09714cbdeba8db23f (127.11.192.195:46369), 149bf9df73814bbabf72c34058df1ed1 (127.11.192.194:38967)
I20260812 07:06:31.191586 9711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92c0ea9d5fb743efb9108415b984c8dc" candidate_uuid: "2fd1bd31c2b44db8af74961163f0f889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa426e29ae2342c09714cbdeba8db23f"
I20260812 07:06:31.191638 9485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92c0ea9d5fb743efb9108415b984c8dc" candidate_uuid: "2fd1bd31c2b44db8af74961163f0f889" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "149bf9df73814bbabf72c34058df1ed1"
I20260812 07:06:31.191761 9711 raft_consensus.cc:3037] T 92c0ea9d5fb743efb9108415b984c8dc P fa426e29ae2342c09714cbdeba8db23f [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:06:31.191794 9485 raft_consensus.cc:3037] T 92c0ea9d5fb743efb9108415b984c8dc P 149bf9df73814bbabf72c34058df1ed1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:06:31.193851 9711 raft_consensus.cc:2454] T 92c0ea9d5fb743efb9108415b984c8dc P fa426e29ae2342c09714cbdeba8db23f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd1bd31c2b44db8af74961163f0f889 in term 1.
I20260812 07:06:31.193938 9485 raft_consensus.cc:2454] T 92c0ea9d5fb743efb9108415b984c8dc P 149bf9df73814bbabf72c34058df1ed1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd1bd31c2b44db8af74961163f0f889 in term 1.
I20260812 07:06:31.194221 9133 leader_election.cc:302] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd1bd31c2b44db8af74961163f0f889, fa426e29ae2342c09714cbdeba8db23f; no voters:
I20260812 07:06:31.194495 9831 raft_consensus.cc:2781] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:06:31.194888 9831 raft_consensus.cc:686] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 LEADER]: Becoming Leader. State: Replica: 2fd1bd31c2b44db8af74961163f0f889, State: Running, Role: LEADER
I20260812 07:06:31.195345 9831 consensus_queue.cc:237] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } }
I20260812 07:06:31.199481 8996 catalog_manager.cc:5495] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fd1bd31c2b44db8af74961163f0f889 (127.11.192.193). New cstate: current_term: 1 leader_uuid: "2fd1bd31c2b44db8af74961163f0f889" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } health_report { overall_health: HEALTHY } } }
W20260812 07:06:31.206895 9336 tablet.cc:2369] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:06:31.221957 9817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:31.216904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44800 (local address 127.11.192.193:41097)
0812 07:06:31.217078 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:31.217084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:31.217158 (+ 74us) server_negotiation.cc:407] Connection header received
0812 07:06:31.217367 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:31.217371 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:31.217440 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:31.217561 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:06:31.217918 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:31.218599 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:31.219908 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:31.220341 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:31.220451 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:31.220955 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:06:31.221393 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:06:31.221530 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:31.221641 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 07:06:31.221719 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 07:06:31.229101 9711 raft_consensus.cc:1261] T 92c0ea9d5fb743efb9108415b984c8dc P fa426e29ae2342c09714cbdeba8db23f [term 1 FOLLOWER]: Refusing update from remote peer 2fd1bd31c2b44db8af74961163f0f889: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:06:31.229100 9485 raft_consensus.cc:1261] T 92c0ea9d5fb743efb9108415b984c8dc P 149bf9df73814bbabf72c34058df1ed1 [term 1 FOLLOWER]: Refusing update from remote peer 2fd1bd31c2b44db8af74961163f0f889: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:06:31.229980 9836 consensus_queue.cc:1035] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:06:31.230302 9831 consensus_queue.cc:1035] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:06:31.240406 9846 mvcc.cc:205] Tried to move back new op lower bound from 7317579330465669120 to 7317579330339610624. Current Snapshot: MvccSnapshot[applied={T|T < 7317579330465669120}]
I20260812 07:06:31.253286 8995 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:06:31.253558 8995 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:06:31.256886 8995 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4: Sending DeleteTablet for 3 replicas of tablet 92c0ea9d5fb743efb9108415b984c8dc
I20260812 07:06:31.257944 9681 tablet_service.cc:1514] Processing DeleteTablet for tablet 92c0ea9d5fb743efb9108415b984c8dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:06:31 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:33442
I20260812 07:06:31.258249 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 9058
I20260812 07:06:31.258224 9211 tablet_service.cc:1514] Processing DeleteTablet for tablet 92c0ea9d5fb743efb9108415b984c8dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:06:31 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:44784
I20260812 07:06:31.258754 9869 tablet_replica.cc:330] T 92c0ea9d5fb743efb9108415b984c8dc P fa426e29ae2342c09714cbdeba8db23f: stopping tablet replica
I20260812 07:06:31.258821 9447 tablet_service.cc:1514] Processing DeleteTablet for tablet 92c0ea9d5fb743efb9108415b984c8dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:06:31 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:50222
I20260812 07:06:31.259146 9869 raft_consensus.cc:2229] T 92c0ea9d5fb743efb9108415b984c8dc P fa426e29ae2342c09714cbdeba8db23f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:06:31.259603 9872 tablet_replica.cc:330] T 92c0ea9d5fb743efb9108415b984c8dc P 149bf9df73814bbabf72c34058df1ed1: stopping tablet replica
I20260812 07:06:31.259671 9869 raft_consensus.cc:2258] T 92c0ea9d5fb743efb9108415b984c8dc P fa426e29ae2342c09714cbdeba8db23f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:06:31.260004 9872 raft_consensus.cc:2229] T 92c0ea9d5fb743efb9108415b984c8dc P 149bf9df73814bbabf72c34058df1ed1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:06:31.261018 9872 raft_consensus.cc:2258] T 92c0ea9d5fb743efb9108415b984c8dc P 149bf9df73814bbabf72c34058df1ed1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:06:31.262424 9869 ts_tablet_manager.cc:1905] T 92c0ea9d5fb743efb9108415b984c8dc P fa426e29ae2342c09714cbdeba8db23f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:06:31.265512 9872 ts_tablet_manager.cc:1905] T 92c0ea9d5fb743efb9108415b984c8dc P 149bf9df73814bbabf72c34058df1ed1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:06:31.269464 9869 ts_tablet_manager.cc:1918] T 92c0ea9d5fb743efb9108415b984c8dc P fa426e29ae2342c09714cbdeba8db23f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:06:31.269666 9869 log.cc:1196] T 92c0ea9d5fb743efb9108415b984c8dc P fa426e29ae2342c09714cbdeba8db23f: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/92c0ea9d5fb743efb9108415b984c8dc
I20260812 07:06:31.270210 9869 ts_tablet_manager.cc:1939] T 92c0ea9d5fb743efb9108415b984c8dc P fa426e29ae2342c09714cbdeba8db23f: Deleting consensus metadata
I20260812 07:06:31.271931 8971 catalog_manager.cc:4841] TS fa426e29ae2342c09714cbdeba8db23f (127.11.192.195:46369): tablet 92c0ea9d5fb743efb9108415b984c8dc (table test-table [id=626a525f7fb643bcbc13e9bd71629c5f]) successfully deleted
I20260812 07:06:31.276046 9817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:31.259370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44814 (local address 127.11.192.193:41097)
0812 07:06:31.259524 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:31.259530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:31.259548 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:06:31.259606 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:31.259610 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:31.259682 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:31.259791 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:06:31.261648 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:31.262842 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:31.264800 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:31.265469 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:31.265594 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:31.268924 (+ 3330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:06:31.268937 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:06:31.268952 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:06:31.269022 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:06:31.272709 (+ 3687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:31.273557 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:31.273566 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:31.273570 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:31.273708 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:31.274567 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:31.274572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:31.274576 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:31.274901 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:06:31.275250 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:31.275499 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 07:06:31.275701 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:06:31.276611 9872 ts_tablet_manager.cc:1918] T 92c0ea9d5fb743efb9108415b984c8dc P 149bf9df73814bbabf72c34058df1ed1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:06:31.276731 9327 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:06:31.276822 9872 log.cc:1196] T 92c0ea9d5fb743efb9108415b984c8dc P 149bf9df73814bbabf72c34058df1ed1: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/92c0ea9d5fb743efb9108415b984c8dc
I20260812 07:06:31.277407 9872 ts_tablet_manager.cc:1939] T 92c0ea9d5fb743efb9108415b984c8dc P 149bf9df73814bbabf72c34058df1ed1: Deleting consensus metadata
I20260812 07:06:31.279099 8969 catalog_manager.cc:4841] TS 149bf9df73814bbabf72c34058df1ed1 (127.11.192.194:38967): tablet 92c0ea9d5fb743efb9108415b984c8dc (table test-table [id=626a525f7fb643bcbc13e9bd71629c5f]) successfully deleted
I20260812 07:06:31.371140 9870 tablet_replica.cc:330] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889: stopping tablet replica
I20260812 07:06:31.371184 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 9058
I20260812 07:06:31.399237 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 9339
W20260812 07:06:31.399322 8969 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.192.193:41097 (error 108)
I20260812 07:06:31.400964 9806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:31.400584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:41097 (local address 127.0.0.1:44822)
0812 07:06:31.400726 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:31.400823 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:41097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 07:06:31.401201 8969 catalog_manager.cc:4584] TS 2fd1bd31c2b44db8af74961163f0f889 (127.11.192.193:41097): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 92c0ea9d5fb743efb9108415b984c8dc: Network error: Client connection negotiation failed: client connection to 127.11.192.193:41097: connect: Connection refused (error 111)
I20260812 07:06:31.413316 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:31.400096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50238 (local address 127.11.192.194:38967)
0812 07:06:31.400287 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:31.400291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:31.400475 (+ 184us) server_negotiation.cc:407] Connection header received
0812 07:06:31.400635 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:31.400640 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:31.400738 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:31.400834 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:06:31.402506 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:31.403288 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:31.404674 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:31.405103 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:31.405211 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:31.407676 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:06:31.407687 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:06:31.407699 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:06:31.407739 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:06:31.410621 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:31.411299 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:31.411306 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:31.411309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:31.411441 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:31.411796 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:31.411800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:31.411802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:31.412203 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:06:31.412483 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:31.412770 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 07:06:31.412984 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 07:06:31.413899 9549 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:06:31.454690 9806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:31.454283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:41097 (local address 127.0.0.1:44826)
0812 07:06:31.454478 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:31.454562 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:41097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 07:06:31.455626 9806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:31.455289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:41097 (local address 127.0.0.1:44828)
0812 07:06:31.455438 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:31.455503 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:41097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 07:06:31.502084 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 9339
I20260812 07:06:31.515960 9806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:31.515532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:41097 (local address 127.0.0.1:44842)
0812 07:06:31.515713 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:31.515796 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:41097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 07:06:31.516968 9806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:31.516616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:41097 (local address 127.0.0.1:44856)
0812 07:06:31.516766 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:31.516834 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:41097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 07:06:31.530370 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 9562
I20260812 07:06:31.543903 9811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:31.531076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33444 (local address 127.11.192.195:46369)
0812 07:06:31.531253 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:31.531258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:31.531278 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:06:31.531449 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:31.531453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:31.531520 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:31.531603 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:06:31.532964 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:31.533678 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:31.534984 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:31.535379 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:31.535493 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:31.538553 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:06:31.538571 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:06:31.538594 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:06:31.538640 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:06:31.541407 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:31.542099 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:31.542107 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:31.542109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:31.542265 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:31.542594 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:31.542597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:31.542599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:31.542952 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:06:31.543165 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:31.543403 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 07:06:31.543586 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:06:31.544509 9779 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:06:31.597555 9806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:31.597058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:41097 (local address 127.0.0.1:44866)
0812 07:06:31.597282 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:31.597382 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:41097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260812 07:06:31.598582 9806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:31.598216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:41097 (local address 127.0.0.1:44870)
0812 07:06:31.598402 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:31.598472 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:41097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260812 07:06:31.631314 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 9562
I20260812 07:06:31.654310 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 8938
I20260812 07:06:31.655423 9042 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:06:31.765880 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 8938
I20260812 07:06:51.795885 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:37881
--webserver_interface=127.11.192.254
--webserver_port=46593
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:37881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:06:52.120400 9881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:06:52.120658 9881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:06:52.120784 9881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:06:52.127581 9881 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 07:06:52.127662 9881 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 07:06:52.127725 9881 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 07:06:52.127794 9881 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:06:52.127862 9881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:06:52.127948 9881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:06:52.128010 9881 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:06:52.128098 9881 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:06:52.137322 9881 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:37881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:37881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=46593
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:06:52.138392 9881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:06:52.139690 9881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:06:52.150751 9896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:52.150723 9898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:52.150955 9900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:06:52.151645 9881 server_base.cc:1034] running on GCE node
I20260812 07:06:52.152405 9881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:06:52.153631 9881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:06:52.154857 9881 hybrid_clock.cc:648] HybridClock initialized: now 1786518412154792 us; error 31 us; skew 500 ppm
I20260812 07:06:52.155186 9881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:06:52 dist-test-slave-lgr5 krb5kdc[7935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518412, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:06:52.161015 9881 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:06:52.162385 9881 webserver.cc:460] Webserver started at http://127.11.192.254:46593/ using document root <none> and password file <none>
I20260812 07:06:52.163041 9881 fs_manager.cc:359] Metadata directory not provided
I20260812 07:06:52.163141 9881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:06:52.168332 9881 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:06:52.170895 9910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:52.172184 9881 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:06:52.172333 9881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "4ee74e07b85b420d9a3faea5221e66a4"
format_stamp: "Formatted at 2026-08-12 07:06:06 on dist-test-slave-lgr5"
server_key: "f10d94cc9bbf70796bd69cd8d36a8bff"
server_key_iv: "a485227446a83ca1386e99541ff35f0c"
server_key_version: "encryptionkey@0"
I20260812 07:06:52.173229 9881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:06:52.185607 9881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:06:52.189327 9881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:06:52.189571 9881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:06:52.206694 9881 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:37881
I20260812 07:06:52.206729 9997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:37881 every 8 connection(s)
I20260812 07:06:52.208060 9881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:06:52.212502 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 9881
I20260812 07:06:52.213284 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:41097
--local_ip_for_outbound_sockets=127.11.192.193
--tserver_master_addrs=127.11.192.254:37881
--webserver_port=41117
--webserver_interface=127.11.192.193
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:06:52.214560 10007 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:06:52.217679 10007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4: Bootstrap starting.
I20260812 07:06:52.237187 10007 log.cc:824] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4: Log is configured to *not* fsync() on all Append() calls
I20260812 07:06:52.252957 10007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:06:52.253432 10007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4: Bootstrap complete.
I20260812 07:06:52.262166 10007 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee74e07b85b420d9a3faea5221e66a4" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 37881 } }
I20260812 07:06:52.262998 10007 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee74e07b85b420d9a3faea5221e66a4, State: Initialized, Role: FOLLOWER
I20260812 07:06:52.263599 10007 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee74e07b85b420d9a3faea5221e66a4" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 37881 } }
I20260812 07:06:52.263814 10007 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:06:52.263921 10007 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:06:52.264093 10007 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 2 FOLLOWER]: Advancing to term 3
I20260812 07:06:52.267637 10007 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee74e07b85b420d9a3faea5221e66a4" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 37881 } }
I20260812 07:06:52.268096 10007 leader_election.cc:302] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ee74e07b85b420d9a3faea5221e66a4; no voters:
I20260812 07:06:52.268710 10007 leader_election.cc:288] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 07:06:52.268857 10013 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 07:06:52.269964 10013 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [term 3 LEADER]: Becoming Leader. State: Replica: 4ee74e07b85b420d9a3faea5221e66a4, State: Running, Role: LEADER
I20260812 07:06:52.270561 10013 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee74e07b85b420d9a3faea5221e66a4" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 37881 } }
I20260812 07:06:52.270745 10007 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:06:52.273805 10016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ee74e07b85b420d9a3faea5221e66a4. Latest consensus state: current_term: 3 leader_uuid: "4ee74e07b85b420d9a3faea5221e66a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee74e07b85b420d9a3faea5221e66a4" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 37881 } } }
I20260812 07:06:52.274163 10016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [sys.catalog]: This master's current role is: LEADER
I20260812 07:06:52.274669 10015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "4ee74e07b85b420d9a3faea5221e66a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee74e07b85b420d9a3faea5221e66a4" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 37881 } } }
I20260812 07:06:52.274936 10015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4 [sys.catalog]: This master's current role is: LEADER
I20260812 07:06:52.276952 10026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:06:52.292212 10026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:06:52.293749 10026 catalog_manager.cc:1259] Loaded cluster ID: 8f2470c482394ec7b633fbb5816820f0
I20260812 07:06:52.293896 10026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:06:52.298148 10026 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:06:52.299831 10026 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4: Loaded TSK: 0
I20260812 07:06:52.300046 10026 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4ee74e07b85b420d9a3faea5221e66a4: Loaded TSK: 1
F20260812 07:06:52.301222 10026 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f62237159ad google::LogMessage::Fail() at ??:0
@ 0x7f622371888a google::LogMessage::SendToLog() at ??:0
@ 0x7f62237154de google::LogMessage::Flush() at ??:0
@ 0x7f6223716a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f62274797d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f622746873e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f62391441c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f6239143783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f62391cf2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f62391cf0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f62392195e9 std::function<>::operator()() at ??:0
@ 0x7f6239147cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f6239145c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f62391cd610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f62391cd439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f6226361345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f622636491c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f6226364799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f622633feaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f62296196db start_thread at ??:0
@ 0x7f62225af61f clone at ??:0
W20260812 07:06:52.631500 10010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:06:52.631764 10010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:06:52.631891 10010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:06:52.638849 10010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:06:52.638955 10010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:06:52.639108 10010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:06:52.648334 10010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:41097
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=41117
--tserver_master_addrs=127.11.192.254:37881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:06:52.649433 10010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:06:52.650652 10010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:06:52.662914 10085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:52.662876 10087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:52.662958 10094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:06:52.664176 10010 server_base.cc:1034] running on GCE node
I20260812 07:06:52.664757 10010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:06:52.665592 10010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:06:52.666790 10010 hybrid_clock.cc:648] HybridClock initialized: now 1786518412666741 us; error 38 us; skew 500 ppm
I20260812 07:06:52.667089 10010 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:06:52 dist-test-slave-lgr5 krb5kdc[7935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518412, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:06:52.670044 10010 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:06:52.671322 10010 webserver.cc:460] Webserver started at http://127.11.192.193:41117/ using document root <none> and password file <none>
I20260812 07:06:52.671945 10010 fs_manager.cc:359] Metadata directory not provided
I20260812 07:06:52.672044 10010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:06:52.677103 10010 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:06:52.681001 10104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:52.682212 10010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:06:52.682346 10010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "2fd1bd31c2b44db8af74961163f0f889"
format_stamp: "Formatted at 2026-08-12 07:06:07 on dist-test-slave-lgr5"
server_key: "6c62a0211db307d9bc49b8b898fb0f9c"
server_key_iv: "a528c1d510c621f597de99da4b75fd6f"
server_key_version: "encryptionkey@0"
I20260812 07:06:52.683209 10010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:06:52.734911 10010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:06:52.738931 10010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:06:52.739144 10010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:06:52.740054 10010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:06:52.741714 10117 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 07:06:52.747141 10010 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 07:06:52.747248 10010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260812 07:06:52.747362 10010 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 07:06:52.750352 10010 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 07:06:52.750432 10010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20260812 07:06:52.750725 10117 tablet_bootstrap.cc:492] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889: Bootstrap starting.
I20260812 07:06:52.777211 10117 log.cc:824] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889: Log is configured to *not* fsync() on all Append() calls
I20260812 07:06:52.786254 10010 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:41097
I20260812 07:06:52.786278 10297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:41097 every 8 connection(s)
I20260812 07:06:52.787528 10010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:06:52.788964 10117 tablet_bootstrap.cc:492] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:06:52.789414 10117 tablet_bootstrap.cc:492] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889: Bootstrap complete.
I20260812 07:06:52.791081 10117 ts_tablet_manager.cc:1397] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889: Time spent bootstrapping tablet: real 0.041s user 0.025s sys 0.012s
I20260812 07:06:52.792307 10301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:52.790298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.193:46125)
0812 07:06:52.791087 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:52.791381 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":337,"threads_started":1}
I20260812 07:06:52.792359 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 10010
W20260812 07:06:52.793038 10298 heartbeater.cc:644] Failed to heartbeat to 127.11.192.254:37881 (0 consecutive failures): Network error: Failed to ping master at 127.11.192.254:37881: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
I20260812 07:06:52.793109 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:38967
--local_ip_for_outbound_sockets=127.11.192.194
--tserver_master_addrs=127.11.192.254:37881
--webserver_port=46083
--webserver_interface=127.11.192.194
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:06:52.793853 10301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:52.793524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.193:55321)
0812 07:06:52.793675 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:52.793742 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 07:06:52.794752 10301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:52.794394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.193:45197)
0812 07:06:52.794538 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:52.794623 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 07:06:52.795034 10298 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 07:06:52.799505 10117 raft_consensus.cc:348] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } }
I20260812 07:06:52.800632 10117 raft_consensus.cc:729] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd1bd31c2b44db8af74961163f0f889, State: Initialized, Role: FOLLOWER
I20260812 07:06:52.801231 10117 consensus_queue.cc:260] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } }
I20260812 07:06:52.802620 10301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:52.802309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.193:59859)
0812 07:06:52.802449 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:52.802515 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 07:06:52.802770 10117 ts_tablet_manager.cc:1428] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
I20260812 07:06:52.803588 10301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:52.803281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.193:36087)
0812 07:06:52.803411 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:52.803470 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 07:06:53.124730 10305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:06:53.124991 10305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:06:53.125131 10305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:06:53.132014 10305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:06:53.132138 10305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:06:53.132277 10305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:06:53.141479 10305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:38967
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=46083
--tserver_master_addrs=127.11.192.254:37881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:06:53.142580 10305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:06:53.143821 10305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:06:53.156190 10315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:53.156245 10318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:53.156210 10321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:06:53.157428 10305 server_base.cc:1034] running on GCE node
I20260812 07:06:53.158051 10305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:06:53.158933 10305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:06:53.160151 10305 hybrid_clock.cc:648] HybridClock initialized: now 1786518413160103 us; error 37 us; skew 500 ppm
I20260812 07:06:53.160445 10305 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:06:53 dist-test-slave-lgr5 krb5kdc[7935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518413, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:06:53.163340 10305 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:06:53.164659 10305 webserver.cc:460] Webserver started at http://127.11.192.194:46083/ using document root <none> and password file <none>
I20260812 07:06:53.165254 10305 fs_manager.cc:359] Metadata directory not provided
I20260812 07:06:53.165354 10305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:06:53.170329 10305 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260812 07:06:53.172842 10331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:53.174070 10305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:06:53.174204 10305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "149bf9df73814bbabf72c34058df1ed1"
format_stamp: "Formatted at 2026-08-12 07:06:07 on dist-test-slave-lgr5"
server_key: "71c6818c8664e1c9f1d62c15c87dabc4"
server_key_iv: "b40201d1784c17dc0e4c8f7f140bcb6a"
server_key_version: "encryptionkey@0"
I20260812 07:06:53.175148 10305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:06:53.198448 10305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:06:53.202112 10305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:06:53.202319 10305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:06:53.203195 10305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:06:53.204450 10305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:06:53.204555 10305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:53.204638 10305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:06:53.204695 10305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:53.233505 10305 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:38967
I20260812 07:06:53.233563 10522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:38967 every 8 connection(s)
I20260812 07:06:53.234814 10305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:06:53.239154 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:53.237453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.194:35399)
0812 07:06:53.238136 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:53.238353 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":260,"threads_started":1}
W20260812 07:06:53.239770 10524 heartbeater.cc:644] Failed to heartbeat to 127.11.192.254:37881 (0 consecutive failures): Network error: Failed to ping master at 127.11.192.254:37881: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
I20260812 07:06:53.240547 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:53.240237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.194:40603)
0812 07:06:53.240377 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:53.240445 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 07:06:53.240549 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 10305
I20260812 07:06:53.241243 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:46369
--local_ip_for_outbound_sockets=127.11.192.195
--tserver_master_addrs=127.11.192.254:37881
--webserver_port=36431
--webserver_interface=127.11.192.195
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:06:53.241393 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:53.241085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.194:58775)
0812 07:06:53.241203 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:53.241272 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 07:06:53.241614 10524 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 07:06:53.571409 10528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:06:53.571667 10528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:06:53.571792 10528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:06:53.579073 10528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:06:53.579172 10528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:06:53.579316 10528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:06:53.589043 10528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:46369
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=36431
--tserver_master_addrs=127.11.192.254:37881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:06:53.590224 10528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:06:53.591651 10528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:06:53.604885 10537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:53.605235 10536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:06:53.605430 10539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:06:53.606355 10528 server_base.cc:1034] running on GCE node
I20260812 07:06:53.606930 10528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:06:53.607861 10528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:06:53.609110 10528 hybrid_clock.cc:648] HybridClock initialized: now 1786518413609048 us; error 56 us; skew 500 ppm
I20260812 07:06:53.609411 10528 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:06:53 dist-test-slave-lgr5 krb5kdc[7935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518413, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:06:53.612533 10528 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:06:53.613957 10528 webserver.cc:460] Webserver started at http://127.11.192.195:36431/ using document root <none> and password file <none>
I20260812 07:06:53.614575 10528 fs_manager.cc:359] Metadata directory not provided
I20260812 07:06:53.614691 10528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:06:53.619854 10528 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:06:53.622568 10547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:53.624018 10528 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:06:53.624219 10528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "fa426e29ae2342c09714cbdeba8db23f"
format_stamp: "Formatted at 2026-08-12 07:06:08 on dist-test-slave-lgr5"
server_key: "9fb6cdf0a648a4ff9b3c03c008a135e3"
server_key_iv: "ecabdf9367798e0481d0a46251b5b0e7"
server_key_version: "encryptionkey@0"
I20260812 07:06:53.625154 10528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:06:53.657251 10528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:06:53.662314 10528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:06:53.662616 10528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:06:53.663727 10528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:06:53.665337 10528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:06:53.665498 10528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260812 07:06:53.665627 10528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:06:53.665720 10528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:06:53.702028 10528 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:46369
I20260812 07:06:53.702087 10721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:46369 every 8 connection(s)
I20260812 07:06:53.703434 10528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:06:53.708535 10727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:53.706674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.195:39559)
0812 07:06:53.707447 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:53.707665 (+ 218us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":318,"threads_started":1}
W20260812 07:06:53.709247 10722 heartbeater.cc:644] Failed to heartbeat to 127.11.192.254:37881 (0 consecutive failures): Network error: Failed to ping master at 127.11.192.254:37881: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
I20260812 07:06:53.709563 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 10528
I20260812 07:06:53.710167 10727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:53.709793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.195:42853)
0812 07:06:53.709956 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:53.710048 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 07:06:53.711524 10730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:53.710609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.195:49197)
0812 07:06:53.711141 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:53.711272 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":245,"threads_started":1}
W20260812 07:06:53.711768 10722 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 07:06:53.804971 10732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:53.804306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.193:59243)
0812 07:06:53.804711 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:53.804786 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":184,"threads_started":1}
I20260812 07:06:54.242770 10734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:54.242042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.194:36833)
0812 07:06:54.242452 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:54.242549 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":189,"threads_started":1}
I20260812 07:06:54.372241 10735 raft_consensus.cc:482] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:06:54.372522 10735 raft_consensus.cc:504] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } }
I20260812 07:06:54.374081 10735 leader_election.cc:288] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa426e29ae2342c09714cbdeba8db23f (127.11.192.195:46369), 149bf9df73814bbabf72c34058df1ed1 (127.11.192.194:38967)
Aug 12 07:06:54 dist-test-slave-lgr5 krb5kdc[7935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518412, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.195@KRBTEST.COM
Aug 12 07:06:54 dist-test-slave-lgr5 krb5kdc[7935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518412, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.194@KRBTEST.COM
I20260812 07:06:54.412588 10737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:54.374386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:46369 (local address 127.11.192.193:37347)
0812 07:06:54.389537 (+ 15151us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:54.389593 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 07:06:54.390307 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:06:54.393497 (+ 3190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:06:54.393502 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:06:54.394653 (+ 1151us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:06:54.395825 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:54.395855 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:54.398113 (+ 2258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:54.398117 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:54.400400 (+ 2283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:54.400416 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:54.401704 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:54.401711 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:54.402158 (+ 447us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:54.402802 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:06:54.402827 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:06:54.405658 (+ 2831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:06:54.408887 (+ 3229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:06:54.408896 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:06:54.408910 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:06:54.409514 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:06:54.410002 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:06:54.410007 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:06:54.410010 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:06:54.410245 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:06:54.411035 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:06:54.411043 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:06:54.411353 (+ 310us) client_negotiation.cc:769] Sending connection context
0812 07:06:54.411593 (+ 240us) client_negotiation.cc:240] Negotiation successful
0812 07:06:54.412131 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15012,"mutex_wait_us":245,"spinlock_wait_cycles":49664,"thread_start_us":14846,"threads_started":1}
I20260812 07:06:54.412618 10740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:54.374859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:37083 (local address 127.11.192.194:38967)
0812 07:06:54.390377 (+ 15518us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:54.390388 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:54.390428 (+ 40us) server_negotiation.cc:407] Connection header received
0812 07:06:54.392648 (+ 2220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:54.392658 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:54.393000 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:54.393325 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:06:54.396035 (+ 2710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:54.397910 (+ 1875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:54.400536 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:54.401519 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:54.401704 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:54.406202 (+ 4498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:06:54.406210 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:06:54.406221 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:06:54.406255 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:06:54.409126 (+ 2871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:54.409870 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:54.409876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:54.409881 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:54.410080 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:54.410490 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:54.410493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:54.410495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:54.410950 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:06:54.411115 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:54.411858 (+ 743us) server_negotiation.cc:299] Negotiation successful
0812 07:06:54.412236 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15353,"thread_start_us":15634,"threads_started":1}
I20260812 07:06:54.412611 10738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:54.374611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:37347 (local address 127.11.192.195:46369)
0812 07:06:54.390010 (+ 15399us) server_negotiation.cc:206] Beginning negotiation
0812 07:06:54.390027 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 07:06:54.390077 (+ 50us) server_negotiation.cc:407] Connection header received
0812 07:06:54.392449 (+ 2372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:06:54.392460 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:06:54.392942 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:06:54.393325 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:06:54.396035 (+ 2710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:54.397910 (+ 1875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:54.400543 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:06:54.401510 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:06:54.401704 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:54.405949 (+ 4245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:06:54.405961 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:06:54.405978 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:06:54.406037 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:06:54.408745 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:54.409658 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:54.409668 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:54.409675 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:54.409871 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:06:54.410360 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:06:54.410364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:06:54.410367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:06:54.410868 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:06:54.411026 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 07:06:54.411851 (+ 825us) server_negotiation.cc:299] Negotiation successful
0812 07:06:54.412203 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15174,"thread_start_us":15483,"threads_started":1}
I20260812 07:06:54.412657 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:54.374421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:38967 (local address 127.11.192.193:37083)
0812 07:06:54.389859 (+ 15438us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:54.389875 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:06:54.390171 (+ 296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:06:54.393489 (+ 3318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:06:54.393500 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:06:54.394652 (+ 1152us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:06:54.395825 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:54.395855 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:54.398109 (+ 2254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:54.398115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:54.400399 (+ 2284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:06:54.400416 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:06:54.401747 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:06:54.401755 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:06:54.402027 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:06:54.402941 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:06:54.402959 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:06:54.405949 (+ 2990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:06:54.409249 (+ 3300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:06:54.409252 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:06:54.409255 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:06:54.409739 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:06:54.410198 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:06:54.410201 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:06:54.410203 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:06:54.410375 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:06:54.411088 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:06:54.411091 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:06:54.411350 (+ 259us) client_negotiation.cc:769] Sending connection context
0812 07:06:54.411596 (+ 246us) client_negotiation.cc:240] Negotiation successful
0812 07:06:54.412258 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15328,"mutex_wait_us":511,"spinlock_wait_cycles":9216,"thread_start_us":15696,"threads_started":1}
W20260812 07:06:54.416589 10108 leader_election.cc:341] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa426e29ae2342c09714cbdeba8db23f (127.11.192.195:46369): Not found: Tablet not found: 92c0ea9d5fb743efb9108415b984c8dc
W20260812 07:06:54.416839 10107 leader_election.cc:341] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 149bf9df73814bbabf72c34058df1ed1 (127.11.192.194:38967): Not found: Tablet not found: 92c0ea9d5fb743efb9108415b984c8dc
I20260812 07:06:54.416960 10107 leader_election.cc:302] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2fd1bd31c2b44db8af74961163f0f889; no voters: 149bf9df73814bbabf72c34058df1ed1, fa426e29ae2342c09714cbdeba8db23f
I20260812 07:06:54.417322 10735 raft_consensus.cc:2726] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 07:06:54.713011 10746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:54.712217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.195:51991)
0812 07:06:54.712660 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:54.712797 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":202,"threads_started":1}
I20260812 07:06:54.806085 10737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:54.805683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.193:49453)
0812 07:06:54.805847 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:54.805948 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 07:06:55.244305 10747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:55.243601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.194:39901)
0812 07:06:55.244042 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:55.244151 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":246,"threads_started":1}
I20260812 07:06:55.714444 10749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:55.713796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.195:35345)
0812 07:06:55.714220 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:55.714297 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":235,"threads_started":1}
I20260812 07:06:55.807651 10750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:55.806863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.193:56949)
0812 07:06:55.807373 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:55.807450 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":251,"threads_started":1}
I20260812 07:06:56.245712 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:56.245056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.194:58401)
0812 07:06:56.245460 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:56.245547 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":211,"threads_started":1}
I20260812 07:06:56.573434 10752 raft_consensus.cc:482] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:06:56.573621 10752 raft_consensus.cc:504] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } }
I20260812 07:06:56.574251 10752 leader_election.cc:288] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa426e29ae2342c09714cbdeba8db23f (127.11.192.195:46369), 149bf9df73814bbabf72c34058df1ed1 (127.11.192.194:38967)
W20260812 07:06:56.575223 10108 leader_election.cc:341] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa426e29ae2342c09714cbdeba8db23f (127.11.192.195:46369): Not found: Tablet not found: 92c0ea9d5fb743efb9108415b984c8dc
W20260812 07:06:56.575403 10107 leader_election.cc:341] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 149bf9df73814bbabf72c34058df1ed1 (127.11.192.194:38967): Not found: Tablet not found: 92c0ea9d5fb743efb9108415b984c8dc
I20260812 07:06:56.575531 10107 leader_election.cc:302] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2fd1bd31c2b44db8af74961163f0f889; no voters: 149bf9df73814bbabf72c34058df1ed1, fa426e29ae2342c09714cbdeba8db23f
I20260812 07:06:56.575764 10752 raft_consensus.cc:2726] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 07:06:56.715790 10753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:56.715172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.195:52287)
0812 07:06:56.715568 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:56.715644 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":194,"threads_started":1}
I20260812 07:06:56.809202 10754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:56.808448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.193:47039)
0812 07:06:56.808887 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:56.809010 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":221,"threads_started":1}
I20260812 07:06:57.247148 10756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:57.246473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.194:60047)
0812 07:06:57.246874 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:57.246952 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":192,"threads_started":1}
I20260812 07:06:57.717216 10757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:57.716518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.195:33643)
0812 07:06:57.716954 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:57.717047 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":203,"threads_started":1}
I20260812 07:06:57.810736 10758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:57.810091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.193:49287)
0812 07:06:57.810503 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:57.810578 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":212,"threads_started":1}
I20260812 07:06:58.248608 10765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:58.247931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.194:54187)
0812 07:06:58.248361 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:58.248438 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":234,"threads_started":1}
I20260812 07:06:58.718642 10766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:58.717991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.195:58077)
0812 07:06:58.718400 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:58.718479 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":223,"threads_started":1}
I20260812 07:06:58.812150 10767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:58.811461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.193:56459)
0812 07:06:58.811871 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:58.811948 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":200,"threads_started":1}
I20260812 07:06:58.869969 10771 raft_consensus.cc:482] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:06:58.870147 10771 raft_consensus.cc:504] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } }
I20260812 07:06:58.870918 10771 leader_election.cc:288] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa426e29ae2342c09714cbdeba8db23f (127.11.192.195:46369), 149bf9df73814bbabf72c34058df1ed1 (127.11.192.194:38967)
W20260812 07:06:58.871806 10108 leader_election.cc:341] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa426e29ae2342c09714cbdeba8db23f (127.11.192.195:46369): Not found: Tablet not found: 92c0ea9d5fb743efb9108415b984c8dc
W20260812 07:06:58.871979 10107 leader_election.cc:341] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 149bf9df73814bbabf72c34058df1ed1 (127.11.192.194:38967): Not found: Tablet not found: 92c0ea9d5fb743efb9108415b984c8dc
I20260812 07:06:58.872120 10107 leader_election.cc:302] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2fd1bd31c2b44db8af74961163f0f889; no voters: 149bf9df73814bbabf72c34058df1ed1, fa426e29ae2342c09714cbdeba8db23f
I20260812 07:06:58.872344 10771 raft_consensus.cc:2726] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 07:06:59.250111 10772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:59.249422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.194:34223)
0812 07:06:59.249850 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:59.249927 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":226,"threads_started":1}
I20260812 07:06:59.720192 10773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:59.719417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.195:46683)
0812 07:06:59.719860 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:59.719950 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":260,"threads_started":1}
I20260812 07:06:59.813552 10775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:06:59.812892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.193:35793)
0812 07:06:59.813276 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0812 07:06:59.813369 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":196,"threads_started":1}
I20260812 07:07:00.251652 10777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:00.250868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.194:33871)
0812 07:07:00.251375 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:00.251460 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":234,"threads_started":1}
I20260812 07:07:00.721769 10778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:00.720997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.195:47747)
0812 07:07:00.721474 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:00.721581 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":214,"threads_started":1}
I20260812 07:07:00.814937 10779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:00.814284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.193:58967)
0812 07:07:00.814692 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:00.814770 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":207,"threads_started":1}
I20260812 07:07:01.253130 10781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:01.252457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.194:37111)
0812 07:07:01.252860 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:01.252950 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":206,"threads_started":1}
I20260812 07:07:01.723242 10782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:01.722597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.195:43879)
0812 07:07:01.722992 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:01.723072 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":199,"threads_started":1}
I20260812 07:07:01.816391 10783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:01.815704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.193:52979)
0812 07:07:01.816147 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:01.816224 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":212,"threads_started":1}
I20260812 07:07:02.254602 10786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:02.253919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.194:43265)
0812 07:07:02.254359 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:02.254445 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":212,"threads_started":1}
I20260812 07:07:02.724793 10787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:02.724000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.195:33541)
0812 07:07:02.724491 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:02.724606 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":266,"threads_started":1}
I20260812 07:07:02.817889 10788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:02.817225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.193:57587)
0812 07:07:02.817650 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:02.817730 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":206,"threads_started":1}
I20260812 07:07:02.967334 10790 raft_consensus.cc:482] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:07:02.967521 10790 raft_consensus.cc:504] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } }
I20260812 07:07:02.968242 10790 leader_election.cc:288] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa426e29ae2342c09714cbdeba8db23f (127.11.192.195:46369), 149bf9df73814bbabf72c34058df1ed1 (127.11.192.194:38967)
W20260812 07:07:02.969087 10107 leader_election.cc:341] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 149bf9df73814bbabf72c34058df1ed1 (127.11.192.194:38967): Not found: Tablet not found: 92c0ea9d5fb743efb9108415b984c8dc
W20260812 07:07:02.969244 10108 leader_election.cc:341] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa426e29ae2342c09714cbdeba8db23f (127.11.192.195:46369): Not found: Tablet not found: 92c0ea9d5fb743efb9108415b984c8dc
I20260812 07:07:02.969395 10108 leader_election.cc:302] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2fd1bd31c2b44db8af74961163f0f889; no voters: 149bf9df73814bbabf72c34058df1ed1, fa426e29ae2342c09714cbdeba8db23f
I20260812 07:07:02.969635 10790 raft_consensus.cc:2726] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 07:07:03.256054 10791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:03.255373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.194:33173)
0812 07:07:03.255789 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:03.255866 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":217,"threads_started":1}
I20260812 07:07:03.726284 10792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:03.725605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.195:59729)
0812 07:07:03.726021 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:03.726108 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":220,"threads_started":1}
I20260812 07:07:03.819345 10793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:03.818703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.193:34765)
0812 07:07:03.819094 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:03.819171 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":186,"threads_started":1}
I20260812 07:07:04.257521 10794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:04.256859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.194:48779)
0812 07:07:04.257280 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:04.257356 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":203,"threads_started":1}
I20260812 07:07:04.727780 10795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:04.727072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.195:53397)
0812 07:07:04.727518 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:04.727616 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":213,"threads_started":1}
I20260812 07:07:04.820781 10796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:04.820073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.193:51349)
0812 07:07:04.820534 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:04.820623 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":226,"threads_started":1}
I20260812 07:07:05.106135 10797 raft_consensus.cc:482] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:07:05.106320 10797 raft_consensus.cc:504] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } }
I20260812 07:07:05.107005 10797 leader_election.cc:288] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa426e29ae2342c09714cbdeba8db23f (127.11.192.195:46369), 149bf9df73814bbabf72c34058df1ed1 (127.11.192.194:38967)
W20260812 07:07:05.107921 10108 leader_election.cc:341] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa426e29ae2342c09714cbdeba8db23f (127.11.192.195:46369): Not found: Tablet not found: 92c0ea9d5fb743efb9108415b984c8dc
W20260812 07:07:05.108103 10107 leader_election.cc:341] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 149bf9df73814bbabf72c34058df1ed1 (127.11.192.194:38967): Not found: Tablet not found: 92c0ea9d5fb743efb9108415b984c8dc
I20260812 07:07:05.108256 10107 leader_election.cc:302] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2fd1bd31c2b44db8af74961163f0f889; no voters: 149bf9df73814bbabf72c34058df1ed1, fa426e29ae2342c09714cbdeba8db23f
I20260812 07:07:05.108512 10797 raft_consensus.cc:2726] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 07:07:05.258987 10798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:05.258309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.194:35133)
0812 07:07:05.258724 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:05.258806 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":211,"threads_started":1}
I20260812 07:07:05.729391 10800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:05.728576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.195:43787)
0812 07:07:05.729056 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:05.729151 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":235,"threads_started":1}
I20260812 07:07:05.822252 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:05.821570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.193:44971)
0812 07:07:05.822002 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:05.822080 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":218,"threads_started":1}
I20260812 07:07:06.260491 10814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:06.259769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.194:36689)
0812 07:07:06.260247 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:06.260336 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":232,"threads_started":1}
I20260812 07:07:06.730887 10815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:06.730176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.195:55095)
0812 07:07:06.730624 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:06.730727 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":227,"threads_started":1}
I20260812 07:07:06.823731 10817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:06.823030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.193:55405)
0812 07:07:06.823460 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:06.823538 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":214,"threads_started":1}
I20260812 07:07:07.262027 10818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:07.261278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.194:44907)
0812 07:07:07.261745 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:07.261833 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":243,"threads_started":1}
I20260812 07:07:07.732455 10819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:07.731630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.195:39607)
0812 07:07:07.732066 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:07.732180 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":233,"threads_started":1}
I20260812 07:07:07.825484 10820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:07.824792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.193:47433)
0812 07:07:07.825226 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:07.825314 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":221,"threads_started":1}
I20260812 07:07:08.263676 10824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:08.262950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.194:53773)
0812 07:07:08.263409 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:08.263523 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":216,"threads_started":1}
I20260812 07:07:08.607960 10825 raft_consensus.cc:482] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:07:08.608218 10825 raft_consensus.cc:504] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa426e29ae2342c09714cbdeba8db23f" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 46369 } } peers { permanent_uuid: "149bf9df73814bbabf72c34058df1ed1" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38967 } } peers { permanent_uuid: "2fd1bd31c2b44db8af74961163f0f889" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41097 } }
I20260812 07:07:08.608901 10825 leader_election.cc:288] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa426e29ae2342c09714cbdeba8db23f (127.11.192.195:46369), 149bf9df73814bbabf72c34058df1ed1 (127.11.192.194:38967)
W20260812 07:07:08.609858 10108 leader_election.cc:341] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa426e29ae2342c09714cbdeba8db23f (127.11.192.195:46369): Not found: Tablet not found: 92c0ea9d5fb743efb9108415b984c8dc
W20260812 07:07:08.610050 10107 leader_election.cc:341] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 149bf9df73814bbabf72c34058df1ed1 (127.11.192.194:38967): Not found: Tablet not found: 92c0ea9d5fb743efb9108415b984c8dc
I20260812 07:07:08.610175 10107 leader_election.cc:302] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2fd1bd31c2b44db8af74961163f0f889; no voters: 149bf9df73814bbabf72c34058df1ed1, fa426e29ae2342c09714cbdeba8db23f
I20260812 07:07:08.610424 10825 raft_consensus.cc:2726] T 92c0ea9d5fb743efb9108415b984c8dc P 2fd1bd31c2b44db8af74961163f0f889 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 07:07:08.720345 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 10010
I20260812 07:07:08.734113 10828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:08.733415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:37881 (local address 127.11.192.195:55293)
0812 07:07:08.733864 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:08.733952 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.254:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":235,"threads_started":1}
I20260812 07:07:08.734717 10826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:08.721424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55462 (local address 127.11.192.193:41097)
0812 07:07:08.722207 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:08.722214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:08.722240 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:07:08.722300 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:08.722307 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:08.722514 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:08.722672 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:08.724019 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:08.725055 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:08.726304 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:08.726860 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:08.726988 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:08.729552 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:08.729565 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:08.729579 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:08.729611 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:08.732110 (+ 2499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:08.732743 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:08.732750 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:08.732753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:08.732881 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:08.733246 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:08.733251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:08.733254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:08.733586 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:08.733875 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:08.734179 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 07:07:08.734425 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":429,"threads_started":1}
I20260812 07:07:08.735870 10287 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:08.825783 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 10010
I20260812 07:07:08.848354 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 10305
I20260812 07:07:08.860921 10833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:08.849088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59506 (local address 127.11.192.194:38967)
0812 07:07:08.849594 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:08.849598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:08.849613 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:07:08.849668 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:08.849672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:08.849729 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:08.849833 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:08.851183 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:08.851998 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:08.853296 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:08.853730 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:08.853857 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:08.856361 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:08.856366 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:08.856370 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:08.856401 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:08.858710 (+ 2309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:08.859281 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:08.859284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:08.859287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:08.859398 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:08.859691 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:08.859694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:08.859696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:08.859938 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:08.860166 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:08.860456 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 07:07:08.860635 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":245,"threads_started":1}
I20260812 07:07:08.861593 10511 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:08.942957 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 10305
I20260812 07:07:08.971086 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 10528
I20260812 07:07:08.983722 10841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:08.971951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43732 (local address 127.11.192.195:46369)
0812 07:07:08.972639 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:08.972644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:08.972662 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:07:08.972718 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:08.972721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:08.972779 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:08.972865 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:08.974134 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:08.974883 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:08.976194 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:08.976651 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:08.976742 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:08.979131 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:08.979136 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:08.979139 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:08.979164 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:08.981638 (+ 2474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:08.982201 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:08.982204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:08.982207 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:08.982332 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:08.982611 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:08.982614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:08.982615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:08.982855 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:08.983058 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:08.983222 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 07:07:08.983378 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":409,"threads_started":1}
I20260812 07:07:08.984476 10716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:09.065459 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 10528
2026-08-12T07:07:09Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64435 ms)
[----------] 30 tests from SecurityITest (292347 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:07:09 dist-test-slave-lgr5 krb5kdc[10845](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:07:09 dist-test-slave-lgr5 krb5kdc[10845](info): set up 2 sockets
Aug 12 07:07:09 dist-test-slave-lgr5 krb5kdc[10845](info): commencing operation
krb5kdc: starting...
W20260812 07:07:10.348611 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.006s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:07:10 dist-test-slave-lgr5 krb5kdc[10845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:07:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:07:10.599107 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:36941
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:36941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:10.918808 10875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:10.919091 10875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:10.919229 10875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:10.926149 10875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:10.926261 10875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:10.926342 10875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:10.926416 10875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:10.926486 10875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:10.935895 10875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:36941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:36941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:10.937091 10875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:10.938292 10875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:10.949613 10881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:10.949548 10882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:10.949512 10884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:10.950529 10875 server_base.cc:1034] running on GCE node
I20260812 07:07:10.951218 10875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:10.952526 10875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:10.953715 10875 hybrid_clock.cc:648] HybridClock initialized: now 1786518430953673 us; error 29 us; skew 500 ppm
I20260812 07:07:10.954023 10875 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:10 dist-test-slave-lgr5 krb5kdc[10845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518430, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:10.956897 10875 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:07:10.958387 10875 webserver.cc:460] Webserver started at http://127.11.192.254:33059/ using document root <none> and password file <none>
I20260812 07:07:10.959050 10875 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:10.959180 10875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:10.959489 10875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:10.962404 10875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "ae52d48f04574ff986bf08e1104afb19"
format_stamp: "Formatted at 2026-08-12 07:07:10 on dist-test-slave-lgr5"
server_key: "e5c3dbf263daaa53723fab49cd8af152"
server_key_iv: "ecb80c8c1762d736ba3adb22d65167a2"
server_key_version: "encryptionkey@0"
I20260812 07:07:10.963129 10875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "ae52d48f04574ff986bf08e1104afb19"
format_stamp: "Formatted at 2026-08-12 07:07:10 on dist-test-slave-lgr5"
server_key: "e5c3dbf263daaa53723fab49cd8af152"
server_key_iv: "ecb80c8c1762d736ba3adb22d65167a2"
server_key_version: "encryptionkey@0"
I20260812 07:07:10.967985 10875 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260812 07:07:10.971184 10894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:10.972414 10875 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:07:10.972577 10875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "ae52d48f04574ff986bf08e1104afb19"
format_stamp: "Formatted at 2026-08-12 07:07:10 on dist-test-slave-lgr5"
server_key: "e5c3dbf263daaa53723fab49cd8af152"
server_key_iv: "ecb80c8c1762d736ba3adb22d65167a2"
server_key_version: "encryptionkey@0"
I20260812 07:07:10.972741 10875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:10.993484 10875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:10.997044 10875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:10.997287 10875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:11.014225 10875 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:36941
I20260812 07:07:11.014236 10987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:36941 every 8 connection(s)
I20260812 07:07:11.015486 10875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:11.018918 10990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:11.020273 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 10875
I20260812 07:07:11.020462 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:07:11.021093 12035 external_mini_cluster.cc:1428] Setting key cfe9f1d849f0807958158163e7a0db78
I20260812 07:07:11.030156 10990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19: Bootstrap starting.
I20260812 07:07:11.033720 10990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:11.034919 10990 log.cc:824] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19: Log is configured to *not* fsync() on all Append() calls
Aug 12 07:07:11 dist-test-slave-lgr5 krb5kdc[10845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:07:11.037915 10990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19: No bootstrap required, opened a new log
I20260812 07:07:11.044677 10993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:11.022936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54162 (local address 127.11.192.254:36941)
0812 07:07:11.023725 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:11.023738 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:11.023779 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:07:11.025496 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:11.025519 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:11.025858 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:11.026148 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:11.028006 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:11.029317 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:11.031151 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:11.031809 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:11.031927 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:11.036715 (+ 4788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:11.036724 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:11.036741 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:11.036790 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:11.040566 (+ 3776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:11.041164 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:11.041173 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:11.041181 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:11.041384 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:11.041706 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:11.041711 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:11.041714 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:11.042373 (+ 659us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:11.042647 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:11.043006 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 07:07:11.043422 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":358,"threads_started":1}
I20260812 07:07:11.047744 10990 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae52d48f04574ff986bf08e1104afb19" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36941 } }
I20260812 07:07:11.048054 10990 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:11.048156 10990 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae52d48f04574ff986bf08e1104afb19, State: Initialized, Role: FOLLOWER
I20260812 07:07:11.048633 10990 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae52d48f04574ff986bf08e1104afb19" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36941 } }
I20260812 07:07:11.048835 10990 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:11.048933 10990 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:11.049055 10990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:11.051007 10990 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae52d48f04574ff986bf08e1104afb19" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36941 } }
I20260812 07:07:11.051409 10990 leader_election.cc:302] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae52d48f04574ff986bf08e1104afb19; no voters:
I20260812 07:07:11.051913 10990 leader_election.cc:288] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:11.052031 10995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:11.052675 10995 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19 [term 1 LEADER]: Becoming Leader. State: Replica: ae52d48f04574ff986bf08e1104afb19, State: Running, Role: LEADER
I20260812 07:07:11.053223 10995 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae52d48f04574ff986bf08e1104afb19" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36941 } }
I20260812 07:07:11.053936 10990 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:11.056218 10998 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae52d48f04574ff986bf08e1104afb19. Latest consensus state: current_term: 1 leader_uuid: "ae52d48f04574ff986bf08e1104afb19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae52d48f04574ff986bf08e1104afb19" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36941 } } }
I20260812 07:07:11.056639 10998 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:11.056231 10996 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae52d48f04574ff986bf08e1104afb19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae52d48f04574ff986bf08e1104afb19" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36941 } } }
I20260812 07:07:11.057518 10996 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:11.057876 11012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:11.063927 11012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:11.072454 11012 catalog_manager.cc:1347] Generated new cluster ID: 9f71f9127d6e4a1aadd846040c616a9d
I20260812 07:07:11.072554 11012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:11.093745 11012 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:11.094965 11012 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:11.104800 11012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19: Generated new TSK 0
I20260812 07:07:11.105549 11012 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
I20260812 07:07:11.171615 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36941
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:11.497159 11039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:11.497412 11039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:11.497514 11039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:11.504469 11039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:11.504560 11039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:11.504671 11039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:07:11.513811 11039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:11.514878 11039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:11.516170 11039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:11.528267 11044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:11.528241 11045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:11.528198 11047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:11.529040 11039 server_base.cc:1034] running on GCE node
I20260812 07:07:11.529608 11039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:11.530520 11039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:11.531710 11039 hybrid_clock.cc:648] HybridClock initialized: now 1786518431531669 us; error 38 us; skew 500 ppm
I20260812 07:07:11.531991 11039 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:11 dist-test-slave-lgr5 krb5kdc[10845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518431, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:11.534819 11039 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:07:11.536216 11039 webserver.cc:460] Webserver started at http://127.11.192.193:44343/ using document root <none> and password file <none>
I20260812 07:07:11.536826 11039 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:11.536938 11039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:11.537222 11039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:11.539848 11039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "2cc2fcf95cf3471fa4f1fb1269476cfb"
format_stamp: "Formatted at 2026-08-12 07:07:11 on dist-test-slave-lgr5"
server_key: "7c7edc1d42be845e4ca99bd3f10809ea"
server_key_iv: "61e8e0e648505b352ea9d804257cc46f"
server_key_version: "encryptionkey@0"
I20260812 07:07:11.540607 11039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "2cc2fcf95cf3471fa4f1fb1269476cfb"
format_stamp: "Formatted at 2026-08-12 07:07:11 on dist-test-slave-lgr5"
server_key: "7c7edc1d42be845e4ca99bd3f10809ea"
server_key_iv: "61e8e0e648505b352ea9d804257cc46f"
server_key_version: "encryptionkey@0"
I20260812 07:07:11.545348 11039 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 07:07:11.548552 11057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:11.549700 11039 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260812 07:07:11.549846 11039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "2cc2fcf95cf3471fa4f1fb1269476cfb"
format_stamp: "Formatted at 2026-08-12 07:07:11 on dist-test-slave-lgr5"
server_key: "7c7edc1d42be845e4ca99bd3f10809ea"
server_key_iv: "61e8e0e648505b352ea9d804257cc46f"
server_key_version: "encryptionkey@0"
I20260812 07:07:11.550012 11039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:11.569146 11039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:11.572539 11039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:11.572774 11039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:11.573613 11039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:11.574783 11039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:11.574862 11039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:11.574975 11039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:11.575034 11039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:11.604647 11039 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:41397
I20260812 07:07:11.604682 11248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:41397 every 8 connection(s)
I20260812 07:07:11.605764 11039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:07:11.607323 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 11039
I20260812 07:07:11.607517 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:07:11.608402 12035 external_mini_cluster.cc:1428] Setting key 5654f6376894ae746683b1f9db2223c0
Aug 12 07:07:11 dist-test-slave-lgr5 krb5kdc[10845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518431, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:07:11.626303 11252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:11.608181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:48301 (local address 127.11.192.254:36941)
0812 07:07:11.608621 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:11.608625 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:11.609855 (+ 1230us) server_negotiation.cc:407] Connection header received
0812 07:07:11.612200 (+ 2345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:11.612204 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:11.612273 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:11.612357 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:11.614225 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:11.614956 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:11.616946 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:11.617323 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:11.617415 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:11.620987 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:11.620993 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:11.620996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:11.621027 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:11.623328 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:11.624141 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:11.624146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:11.624149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:11.624270 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:11.624697 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:11.624701 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:11.624702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:11.624944 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:11.625076 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:11.625730 (+ 654us) server_negotiation.cc:299] Negotiation successful
0812 07:07:11.625949 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":211,"threads_started":1}
I20260812 07:07:11.626979 11260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:11.608851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:36941 (local address 127.11.192.193:48301)
0812 07:07:11.609688 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:11.609732 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:11.611867 (+ 2135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:11.612616 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:11.612626 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:11.613327 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:11.614064 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:11.614086 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:11.615122 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:11.615127 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:11.616804 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:11.616818 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:11.617532 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:11.617540 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:11.617793 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:11.618432 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:11.618466 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:11.620787 (+ 2321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:11.623454 (+ 2667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:11.623463 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:11.623476 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:11.623964 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:11.624364 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:11.624369 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:11.624372 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:11.624595 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:11.625062 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:11.625067 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:11.625366 (+ 299us) client_negotiation.cc:769] Sending connection context
0812 07:07:11.625578 (+ 212us) client_negotiation.cc:240] Negotiation successful
0812 07:07:11.625914 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":329,"threads_started":1}
I20260812 07:07:11.628337 11250 heartbeater.cc:347] Connected to a master server at 127.11.192.254:36941
I20260812 07:07:11.628639 11250 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:11.629429 11250 heartbeater.cc:511] Master 127.11.192.254:36941 requested a full tablet report, sending...
I20260812 07:07:11.631425 10927 ts_manager.cc:194] Registered new tserver with Master: 2cc2fcf95cf3471fa4f1fb1269476cfb (127.11.192.193:41397)
I20260812 07:07:11.632853 10927 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:48301
WARNING: no policy specified for kudu/127.11.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
I20260812 07:07:11.672310 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36941
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:11.997231 11268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:11.997469 11268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:11.997566 11268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:12.004956 11268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:12.005060 11268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:12.005173 11268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:07:12.014631 11268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:12.015805 11268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:12.017062 11268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:12.029315 11274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:12.029397 11276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:12.029348 11283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:12.030161 11268 server_base.cc:1034] running on GCE node
I20260812 07:07:12.030673 11268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:12.031481 11268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:12.032666 11268 hybrid_clock.cc:648] HybridClock initialized: now 1786518432032615 us; error 43 us; skew 500 ppm
I20260812 07:07:12.032953 11268 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:12 dist-test-slave-lgr5 krb5kdc[10845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518432, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:12.035787 11268 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:07:12.037143 11268 webserver.cc:460] Webserver started at http://127.11.192.194:33597/ using document root <none> and password file <none>
I20260812 07:07:12.037750 11268 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:12.037871 11268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:12.038159 11268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:12.040874 11268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "6ea10533f0bc45b9925dd8cc4100973d"
format_stamp: "Formatted at 2026-08-12 07:07:12 on dist-test-slave-lgr5"
server_key: "830e091491f7198dd9a240648631c606"
server_key_iv: "b23af41eb7fe3f7f91a699abf20432d2"
server_key_version: "encryptionkey@0"
I20260812 07:07:12.041577 11268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "6ea10533f0bc45b9925dd8cc4100973d"
format_stamp: "Formatted at 2026-08-12 07:07:12 on dist-test-slave-lgr5"
server_key: "830e091491f7198dd9a240648631c606"
server_key_iv: "b23af41eb7fe3f7f91a699abf20432d2"
server_key_version: "encryptionkey@0"
I20260812 07:07:12.046271 11268 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:07:12.049865 11297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:12.051038 11268 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:07:12.051177 11268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "6ea10533f0bc45b9925dd8cc4100973d"
format_stamp: "Formatted at 2026-08-12 07:07:12 on dist-test-slave-lgr5"
server_key: "830e091491f7198dd9a240648631c606"
server_key_iv: "b23af41eb7fe3f7f91a699abf20432d2"
server_key_version: "encryptionkey@0"
I20260812 07:07:12.051342 11268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:12.077153 11268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:12.080595 11268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:12.080855 11268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:12.081684 11268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:12.082907 11268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:12.082985 11268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:12.083101 11268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:12.083155 11268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:12.112600 11268 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:43975
I20260812 07:07:12.112656 11474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:43975 every 8 connection(s)
I20260812 07:07:12.113761 11268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:07:12.117000 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 11268
I20260812 07:07:12.117168 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:07:12.117657 12035 external_mini_cluster.cc:1428] Setting key a924233ebbdd33a7f3886a4eac1bec2c
Aug 12 07:07:12 dist-test-slave-lgr5 krb5kdc[10845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518432, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:07:12.132819 11252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:12.116158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:38871 (local address 127.11.192.254:36941)
0812 07:07:12.116393 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:12.116398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:12.117550 (+ 1152us) server_negotiation.cc:407] Connection header received
0812 07:07:12.119359 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:12.119365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:12.119441 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:12.119527 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:12.120994 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.121686 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.123156 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.123637 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.123762 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:12.127366 (+ 3604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:12.127371 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:12.127374 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:12.127407 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:12.129861 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:12.130524 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:12.130527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:12.130530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:12.130650 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:12.131065 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:12.131068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:12.131070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:12.131346 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:12.131471 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:12.132260 (+ 789us) server_negotiation.cc:299] Negotiation successful
0812 07:07:12.132493 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 07:07:12.133723 11478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:12.116617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:36941 (local address 127.11.192.194:38871)
0812 07:07:12.117393 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:12.117436 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:12.119100 (+ 1664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:12.119759 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:12.119767 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:12.120299 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:12.120824 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:12.120850 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.121830 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.121834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:12.123016 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:12.123029 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.123879 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.123886 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:12.124172 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:12.124813 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:12.124839 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:12.127152 (+ 2313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:12.129990 (+ 2838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:12.129999 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:12.130013 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:12.130403 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:12.130754 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:12.130759 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:12.130763 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:12.130947 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:12.131482 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:12.131489 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:12.131865 (+ 376us) client_negotiation.cc:769] Sending connection context
0812 07:07:12.132134 (+ 269us) client_negotiation.cc:240] Negotiation successful
0812 07:07:12.132606 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":299,"threads_started":1}
I20260812 07:07:12.134964 11475 heartbeater.cc:347] Connected to a master server at 127.11.192.254:36941
I20260812 07:07:12.135243 11475 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:12.135890 11475 heartbeater.cc:511] Master 127.11.192.254:36941 requested a full tablet report, sending...
I20260812 07:07:12.137296 10927 ts_manager.cc:194] Registered new tserver with Master: 6ea10533f0bc45b9925dd8cc4100973d (127.11.192.194:43975)
I20260812 07:07:12.138103 10927 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:38871
WARNING: no policy specified for kudu/127.11.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
I20260812 07:07:12.178908 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36941
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:12.499137 11481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:12.499415 11481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:12.499557 11481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:12.506609 11481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:12.506723 11481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:12.506873 11481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:07:12.516131 11481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:12.517279 11481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:12.518486 11481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:12.530601 11499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:12.530668 11491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:12.530642 11493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:12.531384 11481 server_base.cc:1034] running on GCE node
I20260812 07:07:12.531908 11481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:12.532771 11481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:12.533968 11481 hybrid_clock.cc:648] HybridClock initialized: now 1786518432533920 us; error 41 us; skew 500 ppm
I20260812 07:07:12.534248 11481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:12 dist-test-slave-lgr5 krb5kdc[10845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518432, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:12.539932 11481 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:07:12.541302 11481 webserver.cc:460] Webserver started at http://127.11.192.195:36857/ using document root <none> and password file <none>
I20260812 07:07:12.541924 11481 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:12.542040 11481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:12.542352 11481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:12.545159 11481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "15f3f16f1f3d480786d63525f7421f9a"
format_stamp: "Formatted at 2026-08-12 07:07:12 on dist-test-slave-lgr5"
server_key: "1e2f9c0187d086d4fc45065a6202881a"
server_key_iv: "ab662cf09fc715e9818ce743047a036b"
server_key_version: "encryptionkey@0"
I20260812 07:07:12.545889 11481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "15f3f16f1f3d480786d63525f7421f9a"
format_stamp: "Formatted at 2026-08-12 07:07:12 on dist-test-slave-lgr5"
server_key: "1e2f9c0187d086d4fc45065a6202881a"
server_key_iv: "ab662cf09fc715e9818ce743047a036b"
server_key_version: "encryptionkey@0"
I20260812 07:07:12.550689 11481 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 07:07:12.554042 11508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:12.555430 11481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:07:12.555574 11481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "15f3f16f1f3d480786d63525f7421f9a"
format_stamp: "Formatted at 2026-08-12 07:07:12 on dist-test-slave-lgr5"
server_key: "1e2f9c0187d086d4fc45065a6202881a"
server_key_iv: "ab662cf09fc715e9818ce743047a036b"
server_key_version: "encryptionkey@0"
I20260812 07:07:12.555745 11481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:12.571728 11481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:12.575369 11481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:12.575626 11481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:12.576620 11481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:12.577888 11481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:12.577981 11481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:12.578100 11481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:12.578158 11481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:12.607805 11481 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:42557
I20260812 07:07:12.607829 11705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:42557 every 8 connection(s)
I20260812 07:07:12.609040 11481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:07:12.613077 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 11481
I20260812 07:07:12.613260 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:07:12.613840 12035 external_mini_cluster.cc:1428] Setting key 3405b62badfaacfed66f2c704828a230
Aug 12 07:07:12 dist-test-slave-lgr5 krb5kdc[10845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518432, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:07:12.627525 11252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:12.611369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:40661 (local address 127.11.192.254:36941)
0812 07:07:12.611530 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:12.611535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:12.612563 (+ 1028us) server_negotiation.cc:407] Connection header received
0812 07:07:12.614898 (+ 2335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:12.614903 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:12.614998 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:12.615113 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:12.616801 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.617645 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.619110 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.619513 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.619600 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:12.622919 (+ 3319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:12.622924 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:12.622927 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:12.622959 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:12.625198 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:12.625802 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:12.625806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:12.625808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:12.625924 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:12.626237 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:12.626240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:12.626242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:12.626498 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:12.626605 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:12.627123 (+ 518us) server_negotiation.cc:299] Negotiation successful
0812 07:07:12.627277 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 07:07:12.628562 11718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:12.611707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:36941 (local address 127.11.192.195:40661)
0812 07:07:12.612398 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:12.612437 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:12.614567 (+ 2130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:12.615361 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:12.615369 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:12.615853 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:12.616634 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:12.616659 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.617785 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.617789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:12.619002 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:12.619014 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.619678 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.619683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:12.619862 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:12.620421 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:12.620447 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:12.622730 (+ 2283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:12.625314 (+ 2584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:12.625321 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:12.625330 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:12.625715 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:12.625996 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:12.625999 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:12.626001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:12.626159 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:12.626583 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:12.626588 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:12.626846 (+ 258us) client_negotiation.cc:769] Sending connection context
0812 07:07:12.627043 (+ 197us) client_negotiation.cc:240] Negotiation successful
0812 07:07:12.627436 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":279,"threads_started":1}
I20260812 07:07:12.629792 11710 heartbeater.cc:347] Connected to a master server at 127.11.192.254:36941
I20260812 07:07:12.630043 11710 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:12.630656 11710 heartbeater.cc:511] Master 127.11.192.254:36941 requested a full tablet report, sending...
I20260812 07:07:12.631933 10927 ts_manager.cc:194] Registered new tserver with Master: 15f3f16f1f3d480786d63525f7421f9a (127.11.192.195:42557)
I20260812 07:07:12.632714 10927 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:40661
I20260812 07:07:12.635887 11250 heartbeater.cc:503] Master 127.11.192.254:36941 was elected leader, sending a full tablet report...
I20260812 07:07:12.640991 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:07:12.656874 11252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:12.645267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54170 (local address 127.11.192.254:36941)
0812 07:07:12.645445 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:12.645452 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:12.645843 (+ 391us) server_negotiation.cc:407] Connection header received
0812 07:07:12.645967 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:12.645971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:12.646025 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:12.646091 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:12.647414 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.648100 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.649317 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.649713 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.649797 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:12.651923 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:12.651927 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:12.651930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:12.651953 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:12.654386 (+ 2433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:12.654993 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:12.654998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:12.655001 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:12.655144 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:12.655510 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:12.655516 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:12.655519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:12.655804 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:12.656044 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:12.656392 (+ 348us) server_negotiation.cc:299] Negotiation successful
0812 07:07:12.656603 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:07:12.661082 10927 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:07:12.667632 10927 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:07:12.688753 11737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:12.679248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:41397 (local address 127.0.0.1:53732)
0812 07:07:12.679790 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:12.679840 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:12.680036 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:12.682334 (+ 2298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:12.682339 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:12.682358 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:12.682737 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:12.682749 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.684590 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.684596 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:12.686407 (+ 1811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:12.686423 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.687935 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.687941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:12.688111 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:12.688147 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 07:07:12.688214 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 07:07:12.688328 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":1,"thread_start_us":627,"threads_started":1}
I20260812 07:07:12.689206 11739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:12.679758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53732 (local address 127.11.192.193:41397)
0812 07:07:12.680983 (+ 1225us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:12.680991 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:12.681015 (+ 24us) server_negotiation.cc:407] Connection header received
0812 07:07:12.681097 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:12.681103 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:12.681314 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:12.681505 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:12.682916 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.684457 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.686558 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.687705 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.688530 (+ 825us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:12.688693 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:12.688763 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 07:07:12.688916 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1056,"thread_start_us":1014,"threads_started":1}
I20260812 07:07:12.688372 11733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:12.678624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:43975 (local address 127.0.0.1:40134)
0812 07:07:12.679790 (+ 1166us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:12.679840 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:12.680036 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:12.681057 (+ 1021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:12.681064 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:12.681101 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:12.681474 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:12.681488 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.683313 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.683319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:12.685606 (+ 2287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:12.685621 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.687358 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.687365 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:12.687564 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:12.687618 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 07:07:12.687778 (+ 160us) client_negotiation.cc:240] Negotiation successful
0812 07:07:12.687964 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"thread_start_us":743,"threads_started":1}
I20260812 07:07:12.689383 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:12.678815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40134 (local address 127.11.192.194:43975)
0812 07:07:12.679312 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:12.679320 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:12.679950 (+ 630us) server_negotiation.cc:407] Connection header received
0812 07:07:12.680307 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:12.680315 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:12.680562 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:12.680761 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:12.681664 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.683177 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.685747 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.687162 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.687319 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:12.687486 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:12.687819 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 07:07:12.688000 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":245,"threads_started":1}
I20260812 07:07:12.691345 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:12.680183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:42557 (local address 127.0.0.1:35526)
0812 07:07:12.682086 (+ 1903us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:12.682102 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:12.682210 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:12.683726 (+ 1516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:12.683730 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:12.683753 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:12.684147 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:12.684163 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.685890 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.685895 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:12.687820 (+ 1925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:12.687835 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.690664 (+ 2829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.690671 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:12.690831 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:12.690862 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 07:07:12.690956 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 07:07:12.691071 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1768,"thread_start_us":1640,"threads_started":1}
I20260812 07:07:12.691483 11743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:12.680611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35526 (local address 127.11.192.195:42557)
0812 07:07:12.683309 (+ 2698us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:12.683317 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:12.683341 (+ 24us) server_negotiation.cc:407] Connection header received
0812 07:07:12.683417 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:12.683422 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:12.683566 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:12.683717 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:12.684305 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.685763 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.688011 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.690450 (+ 2439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.690650 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:12.690804 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:12.691035 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 07:07:12.691189 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2518,"thread_start_us":2373,"threads_started":1}
I20260812 07:07:12.692921 11134 tablet_service.cc:1467] Processing CreateTablet for tablet c092866bf3c644a5a8493fe77ee56b26 (DEFAULT_TABLE table=test-table [id=1a7b24493bca4ac984bc3f1dd346bf16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:12.693717 11374 tablet_service.cc:1467] Processing CreateTablet for tablet c092866bf3c644a5a8493fe77ee56b26 (DEFAULT_TABLE table=test-table [id=1a7b24493bca4ac984bc3f1dd346bf16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:12.694460 11134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c092866bf3c644a5a8493fe77ee56b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:12.694778 11374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c092866bf3c644a5a8493fe77ee56b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:12.695084 11595 tablet_service.cc:1467] Processing CreateTablet for tablet c092866bf3c644a5a8493fe77ee56b26 (DEFAULT_TABLE table=test-table [id=1a7b24493bca4ac984bc3f1dd346bf16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:12.696501 11595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c092866bf3c644a5a8493fe77ee56b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:12.708776 11747 tablet_bootstrap.cc:492] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d: Bootstrap starting.
I20260812 07:07:12.709998 11750 tablet_bootstrap.cc:492] T c092866bf3c644a5a8493fe77ee56b26 P 2cc2fcf95cf3471fa4f1fb1269476cfb: Bootstrap starting.
I20260812 07:07:12.713587 11750 tablet_bootstrap.cc:654] T c092866bf3c644a5a8493fe77ee56b26 P 2cc2fcf95cf3471fa4f1fb1269476cfb: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:12.713586 11747 tablet_bootstrap.cc:654] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:12.714423 11753 tablet_bootstrap.cc:492] T c092866bf3c644a5a8493fe77ee56b26 P 15f3f16f1f3d480786d63525f7421f9a: Bootstrap starting.
I20260812 07:07:12.714815 11747 log.cc:824] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:12.714815 11750 log.cc:824] T c092866bf3c644a5a8493fe77ee56b26 P 2cc2fcf95cf3471fa4f1fb1269476cfb: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:12.717516 11747 tablet_bootstrap.cc:492] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d: No bootstrap required, opened a new log
I20260812 07:07:12.717747 11747 ts_tablet_manager.cc:1397] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260812 07:07:12.717872 11753 tablet_bootstrap.cc:654] T c092866bf3c644a5a8493fe77ee56b26 P 15f3f16f1f3d480786d63525f7421f9a: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:12.717921 11750 tablet_bootstrap.cc:492] T c092866bf3c644a5a8493fe77ee56b26 P 2cc2fcf95cf3471fa4f1fb1269476cfb: No bootstrap required, opened a new log
I20260812 07:07:12.718159 11750 ts_tablet_manager.cc:1397] T c092866bf3c644a5a8493fe77ee56b26 P 2cc2fcf95cf3471fa4f1fb1269476cfb: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 07:07:12.719096 11753 log.cc:824] T c092866bf3c644a5a8493fe77ee56b26 P 15f3f16f1f3d480786d63525f7421f9a: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:12.722527 11753 tablet_bootstrap.cc:492] T c092866bf3c644a5a8493fe77ee56b26 P 15f3f16f1f3d480786d63525f7421f9a: No bootstrap required, opened a new log
I20260812 07:07:12.722779 11753 ts_tablet_manager.cc:1397] T c092866bf3c644a5a8493fe77ee56b26 P 15f3f16f1f3d480786d63525f7421f9a: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260812 07:07:12.725524 11747 raft_consensus.cc:348] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ea10533f0bc45b9925dd8cc4100973d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 43975 } } peers { permanent_uuid: "15f3f16f1f3d480786d63525f7421f9a" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42557 } } peers { permanent_uuid: "2cc2fcf95cf3471fa4f1fb1269476cfb" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41397 } }
I20260812 07:07:12.725867 11747 raft_consensus.cc:374] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:12.725962 11747 raft_consensus.cc:729] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ea10533f0bc45b9925dd8cc4100973d, State: Initialized, Role: FOLLOWER
I20260812 07:07:12.726509 11747 consensus_queue.cc:260] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ea10533f0bc45b9925dd8cc4100973d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 43975 } } peers { permanent_uuid: "15f3f16f1f3d480786d63525f7421f9a" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42557 } } peers { permanent_uuid: "2cc2fcf95cf3471fa4f1fb1269476cfb" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41397 } }
I20260812 07:07:12.727710 11750 raft_consensus.cc:348] T c092866bf3c644a5a8493fe77ee56b26 P 2cc2fcf95cf3471fa4f1fb1269476cfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ea10533f0bc45b9925dd8cc4100973d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 43975 } } peers { permanent_uuid: "15f3f16f1f3d480786d63525f7421f9a" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42557 } } peers { permanent_uuid: "2cc2fcf95cf3471fa4f1fb1269476cfb" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41397 } }
I20260812 07:07:12.728106 11750 raft_consensus.cc:374] T c092866bf3c644a5a8493fe77ee56b26 P 2cc2fcf95cf3471fa4f1fb1269476cfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:12.728247 11750 raft_consensus.cc:729] T c092866bf3c644a5a8493fe77ee56b26 P 2cc2fcf95cf3471fa4f1fb1269476cfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc2fcf95cf3471fa4f1fb1269476cfb, State: Initialized, Role: FOLLOWER
I20260812 07:07:12.728725 11475 heartbeater.cc:503] Master 127.11.192.254:36941 was elected leader, sending a full tablet report...
I20260812 07:07:12.728941 11750 consensus_queue.cc:260] T c092866bf3c644a5a8493fe77ee56b26 P 2cc2fcf95cf3471fa4f1fb1269476cfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ea10533f0bc45b9925dd8cc4100973d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 43975 } } peers { permanent_uuid: "15f3f16f1f3d480786d63525f7421f9a" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42557 } } peers { permanent_uuid: "2cc2fcf95cf3471fa4f1fb1269476cfb" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41397 } }
I20260812 07:07:12.729362 11747 ts_tablet_manager.cc:1428] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260812 07:07:12.732961 11753 raft_consensus.cc:348] T c092866bf3c644a5a8493fe77ee56b26 P 15f3f16f1f3d480786d63525f7421f9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ea10533f0bc45b9925dd8cc4100973d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 43975 } } peers { permanent_uuid: "15f3f16f1f3d480786d63525f7421f9a" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42557 } } peers { permanent_uuid: "2cc2fcf95cf3471fa4f1fb1269476cfb" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41397 } }
I20260812 07:07:12.733381 11753 raft_consensus.cc:374] T c092866bf3c644a5a8493fe77ee56b26 P 15f3f16f1f3d480786d63525f7421f9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:12.733841 11753 raft_consensus.cc:729] T c092866bf3c644a5a8493fe77ee56b26 P 15f3f16f1f3d480786d63525f7421f9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15f3f16f1f3d480786d63525f7421f9a, State: Initialized, Role: FOLLOWER
I20260812 07:07:12.734113 11750 ts_tablet_manager.cc:1428] T c092866bf3c644a5a8493fe77ee56b26 P 2cc2fcf95cf3471fa4f1fb1269476cfb: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260812 07:07:12.734486 11753 consensus_queue.cc:260] T c092866bf3c644a5a8493fe77ee56b26 P 15f3f16f1f3d480786d63525f7421f9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ea10533f0bc45b9925dd8cc4100973d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 43975 } } peers { permanent_uuid: "15f3f16f1f3d480786d63525f7421f9a" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42557 } } peers { permanent_uuid: "2cc2fcf95cf3471fa4f1fb1269476cfb" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41397 } }
I20260812 07:07:12.735414 11710 heartbeater.cc:503] Master 127.11.192.254:36941 was elected leader, sending a full tablet report...
I20260812 07:07:12.736109 11753 ts_tablet_manager.cc:1428] T c092866bf3c644a5a8493fe77ee56b26 P 15f3f16f1f3d480786d63525f7421f9a: Time spent starting tablet: real 0.013s user 0.008s sys 0.006s
I20260812 07:07:12.783910 11764 raft_consensus.cc:482] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:07:12.784204 11764 raft_consensus.cc:504] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ea10533f0bc45b9925dd8cc4100973d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 43975 } } peers { permanent_uuid: "15f3f16f1f3d480786d63525f7421f9a" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42557 } } peers { permanent_uuid: "2cc2fcf95cf3471fa4f1fb1269476cfb" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41397 } }
I20260812 07:07:12.785753 11764 leader_election.cc:288] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15f3f16f1f3d480786d63525f7421f9a (127.11.192.195:42557), 2cc2fcf95cf3471fa4f1fb1269476cfb (127.11.192.193:41397)
I20260812 07:07:12.792459 11743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:12.786137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:44881 (local address 127.11.192.195:42557)
0812 07:07:12.786327 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:12.786334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:12.786815 (+ 481us) server_negotiation.cc:407] Connection header received
0812 07:07:12.786999 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:12.787004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:12.787083 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:12.787173 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:12.787792 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.788755 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.790658 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.791523 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.791638 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:12.791689 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:12.792073 (+ 384us) server_negotiation.cc:299] Negotiation successful
0812 07:07:12.792215 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:07:12.792496 11769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:12.785979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:42557 (local address 127.11.192.194:44881)
0812 07:07:12.786710 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:12.786725 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:12.786860 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:12.787216 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:12.787220 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:12.787252 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:12.787644 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:12.787657 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.788878 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.788884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:12.790500 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:12.790516 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.791730 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.791736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:12.791905 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:12.791940 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 07:07:12.792043 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 07:07:12.792193 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":1136,"threads_started":1}
I20260812 07:07:12.792815 11739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:12.786208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:37629 (local address 127.11.192.193:41397)
0812 07:07:12.786409 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:12.786416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:12.787567 (+ 1151us) server_negotiation.cc:407] Connection header received
0812 07:07:12.787684 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:12.787688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:12.787762 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:12.787834 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:12.788430 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.789261 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.791010 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.791881 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.792031 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:12.792120 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:12.792475 (+ 355us) server_negotiation.cc:299] Negotiation successful
0812 07:07:12.792602 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 07:07:12.792917 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:12.785983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:41397 (local address 127.11.192.194:37629)
0812 07:07:12.787450 (+ 1467us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:12.787464 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:12.787581 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:12.787873 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:12.787878 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:12.787903 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:12.788300 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:12.788313 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.789401 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.789404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:12.790853 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:12.790868 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.792113 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.792121 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:12.792303 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:12.792337 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 07:07:12.792428 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 07:07:12.792564 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1351,"mutex_wait_us":28,"thread_start_us":379,"threads_started":1}
I20260812 07:07:12.793694 11625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c092866bf3c644a5a8493fe77ee56b26" candidate_uuid: "6ea10533f0bc45b9925dd8cc4100973d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15f3f16f1f3d480786d63525f7421f9a" is_pre_election: true
I20260812 07:07:12.793893 11174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c092866bf3c644a5a8493fe77ee56b26" candidate_uuid: "6ea10533f0bc45b9925dd8cc4100973d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc2fcf95cf3471fa4f1fb1269476cfb" is_pre_election: true
I20260812 07:07:12.794263 11625 raft_consensus.cc:2454] T c092866bf3c644a5a8493fe77ee56b26 P 15f3f16f1f3d480786d63525f7421f9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ea10533f0bc45b9925dd8cc4100973d in term 0.
I20260812 07:07:12.794433 11174 raft_consensus.cc:2454] T c092866bf3c644a5a8493fe77ee56b26 P 2cc2fcf95cf3471fa4f1fb1269476cfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ea10533f0bc45b9925dd8cc4100973d in term 0.
I20260812 07:07:12.794950 11302 leader_election.cc:302] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f3f16f1f3d480786d63525f7421f9a, 6ea10533f0bc45b9925dd8cc4100973d; no voters:
I20260812 07:07:12.795349 11764 raft_consensus.cc:2781] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:07:12.795488 11764 raft_consensus.cc:482] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:07:12.795593 11764 raft_consensus.cc:3037] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:12.797720 11764 raft_consensus.cc:504] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ea10533f0bc45b9925dd8cc4100973d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 43975 } } peers { permanent_uuid: "15f3f16f1f3d480786d63525f7421f9a" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42557 } } peers { permanent_uuid: "2cc2fcf95cf3471fa4f1fb1269476cfb" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41397 } }
I20260812 07:07:12.798243 11764 leader_election.cc:288] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d [CANDIDATE]: Term 1 election: Requested vote from peers 15f3f16f1f3d480786d63525f7421f9a (127.11.192.195:42557), 2cc2fcf95cf3471fa4f1fb1269476cfb (127.11.192.193:41397)
I20260812 07:07:12.798637 11174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c092866bf3c644a5a8493fe77ee56b26" candidate_uuid: "6ea10533f0bc45b9925dd8cc4100973d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc2fcf95cf3471fa4f1fb1269476cfb"
I20260812 07:07:12.798662 11625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c092866bf3c644a5a8493fe77ee56b26" candidate_uuid: "6ea10533f0bc45b9925dd8cc4100973d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15f3f16f1f3d480786d63525f7421f9a"
I20260812 07:07:12.798796 11174 raft_consensus.cc:3037] T c092866bf3c644a5a8493fe77ee56b26 P 2cc2fcf95cf3471fa4f1fb1269476cfb [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:12.798815 11625 raft_consensus.cc:3037] T c092866bf3c644a5a8493fe77ee56b26 P 15f3f16f1f3d480786d63525f7421f9a [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:12.800930 11174 raft_consensus.cc:2454] T c092866bf3c644a5a8493fe77ee56b26 P 2cc2fcf95cf3471fa4f1fb1269476cfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ea10533f0bc45b9925dd8cc4100973d in term 1.
I20260812 07:07:12.800951 11625 raft_consensus.cc:2454] T c092866bf3c644a5a8493fe77ee56b26 P 15f3f16f1f3d480786d63525f7421f9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ea10533f0bc45b9925dd8cc4100973d in term 1.
I20260812 07:07:12.801278 11302 leader_election.cc:302] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f3f16f1f3d480786d63525f7421f9a, 6ea10533f0bc45b9925dd8cc4100973d; no voters:
I20260812 07:07:12.801540 11764 raft_consensus.cc:2781] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:12.801976 11764 raft_consensus.cc:686] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d [term 1 LEADER]: Becoming Leader. State: Replica: 6ea10533f0bc45b9925dd8cc4100973d, State: Running, Role: LEADER
I20260812 07:07:12.802449 11764 consensus_queue.cc:237] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ea10533f0bc45b9925dd8cc4100973d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 43975 } } peers { permanent_uuid: "15f3f16f1f3d480786d63525f7421f9a" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42557 } } peers { permanent_uuid: "2cc2fcf95cf3471fa4f1fb1269476cfb" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41397 } }
I20260812 07:07:12.807086 10927 catalog_manager.cc:5495] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ea10533f0bc45b9925dd8cc4100973d (127.11.192.194). New cstate: current_term: 1 leader_uuid: "6ea10533f0bc45b9925dd8cc4100973d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ea10533f0bc45b9925dd8cc4100973d" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 43975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15f3f16f1f3d480786d63525f7421f9a" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cc2fcf95cf3471fa4f1fb1269476cfb" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41397 } health_report { overall_health: UNKNOWN } } }
I20260812 07:07:12.843647 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:12.838848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40144 (local address 127.11.192.194:43975)
0812 07:07:12.839007 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:12.839011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:12.839076 (+ 65us) server_negotiation.cc:407] Connection header received
0812 07:07:12.839253 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:12.839256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:12.839328 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:12.839442 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:07:12.839731 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.840457 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.841764 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.842165 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.842256 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:12.842687 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:07:12.843160 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:07:12.843258 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:12.843335 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 07:07:12.843397 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 07:07:12.850948 11174 raft_consensus.cc:1261] T c092866bf3c644a5a8493fe77ee56b26 P 2cc2fcf95cf3471fa4f1fb1269476cfb [term 1 FOLLOWER]: Refusing update from remote peer 6ea10533f0bc45b9925dd8cc4100973d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:07:12.851001 11625 raft_consensus.cc:1261] T c092866bf3c644a5a8493fe77ee56b26 P 15f3f16f1f3d480786d63525f7421f9a [term 1 FOLLOWER]: Refusing update from remote peer 6ea10533f0bc45b9925dd8cc4100973d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:07:12.851886 11770 consensus_queue.cc:1035] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cc2fcf95cf3471fa4f1fb1269476cfb" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 41397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:07:12.852213 11764 consensus_queue.cc:1035] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d [LEADER]: Connected to new peer: Peer: permanent_uuid: "15f3f16f1f3d480786d63525f7421f9a" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 07:07:12.857918 11251 tablet.cc:2369] T c092866bf3c644a5a8493fe77ee56b26 P 2cc2fcf95cf3471fa4f1fb1269476cfb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:07:12.860461 11711 tablet.cc:2369] T c092866bf3c644a5a8493fe77ee56b26 P 15f3f16f1f3d480786d63525f7421f9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:07:12.864548 11774 mvcc.cc:205] Tried to move back new op lower bound from 7317579500948602880 to 7317579500762611712. Current Snapshot: MvccSnapshot[applied={T|T < 7317579500948602880}]
I20260812 07:07:12.865063 11775 mvcc.cc:205] Tried to move back new op lower bound from 7317579500948602880 to 7317579500762611712. Current Snapshot: MvccSnapshot[applied={T|T < 7317579500948602880}]
I20260812 07:07:12.876493 10927 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:07:12.876765 10927 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:07:12.880244 10927 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae52d48f04574ff986bf08e1104afb19: Sending DeleteTablet for 3 replicas of tablet c092866bf3c644a5a8493fe77ee56b26
I20260812 07:07:12.881328 11134 tablet_service.cc:1514] Processing DeleteTablet for tablet c092866bf3c644a5a8493fe77ee56b26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:12 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:53732
I20260812 07:07:12.881368 11374 tablet_service.cc:1514] Processing DeleteTablet for tablet c092866bf3c644a5a8493fe77ee56b26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:12 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:40134
I20260812 07:07:12.881412 11595 tablet_service.cc:1514] Processing DeleteTablet for tablet c092866bf3c644a5a8493fe77ee56b26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:12 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:35526
I20260812 07:07:12.882153 11784 tablet_replica.cc:330] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d: stopping tablet replica
I20260812 07:07:12.882524 11784 raft_consensus.cc:2229] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:07:12.883193 11784 raft_consensus.cc:2258] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:12.883441 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 11039
I20260812 07:07:12.885267 11784 ts_tablet_manager.cc:1905] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:12.892665 11786 tablet_replica.cc:330] T c092866bf3c644a5a8493fe77ee56b26 P 15f3f16f1f3d480786d63525f7421f9a: stopping tablet replica
I20260812 07:07:12.892841 11784 ts_tablet_manager.cc:1918] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:07:12.893064 11784 log.cc:1196] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/c092866bf3c644a5a8493fe77ee56b26
I20260812 07:07:12.893177 11786 raft_consensus.cc:2229] T c092866bf3c644a5a8493fe77ee56b26 P 15f3f16f1f3d480786d63525f7421f9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:07:12.893657 11784 ts_tablet_manager.cc:1939] T c092866bf3c644a5a8493fe77ee56b26 P 6ea10533f0bc45b9925dd8cc4100973d: Deleting consensus metadata
I20260812 07:07:12.893730 11786 raft_consensus.cc:2258] T c092866bf3c644a5a8493fe77ee56b26 P 15f3f16f1f3d480786d63525f7421f9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:12.894513 11785 tablet_replica.cc:330] T c092866bf3c644a5a8493fe77ee56b26 P 2cc2fcf95cf3471fa4f1fb1269476cfb: stopping tablet replica
I20260812 07:07:12.895509 10898 catalog_manager.cc:4841] TS 6ea10533f0bc45b9925dd8cc4100973d (127.11.192.194:43975): tablet c092866bf3c644a5a8493fe77ee56b26 (table test-table [id=1a7b24493bca4ac984bc3f1dd346bf16]) successfully deleted
I20260812 07:07:12.896116 11785 raft_consensus.cc:2229] T c092866bf3c644a5a8493fe77ee56b26 P 2cc2fcf95cf3471fa4f1fb1269476cfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:07:12.896644 11785 raft_consensus.cc:2258] T c092866bf3c644a5a8493fe77ee56b26 P 2cc2fcf95cf3471fa4f1fb1269476cfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:12.896795 11786 ts_tablet_manager.cc:1905] T c092866bf3c644a5a8493fe77ee56b26 P 15f3f16f1f3d480786d63525f7421f9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:12.898633 11785 ts_tablet_manager.cc:1905] T c092866bf3c644a5a8493fe77ee56b26 P 2cc2fcf95cf3471fa4f1fb1269476cfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:12.903498 11786 ts_tablet_manager.cc:1918] T c092866bf3c644a5a8493fe77ee56b26 P 15f3f16f1f3d480786d63525f7421f9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:07:12.903715 11786 log.cc:1196] T c092866bf3c644a5a8493fe77ee56b26 P 15f3f16f1f3d480786d63525f7421f9a: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/c092866bf3c644a5a8493fe77ee56b26
I20260812 07:07:12.904332 11786 ts_tablet_manager.cc:1939] T c092866bf3c644a5a8493fe77ee56b26 P 15f3f16f1f3d480786d63525f7421f9a: Deleting consensus metadata
I20260812 07:07:12.905270 11785 ts_tablet_manager.cc:1918] T c092866bf3c644a5a8493fe77ee56b26 P 2cc2fcf95cf3471fa4f1fb1269476cfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:07:12.905479 11785 log.cc:1196] T c092866bf3c644a5a8493fe77ee56b26 P 2cc2fcf95cf3471fa4f1fb1269476cfb: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518136653111-12035-0/minicluster-data/ts-0/wal/wals/c092866bf3c644a5a8493fe77ee56b26
Aug 12 07:07:12 dist-test-slave-lgr5 krb5kdc[10845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.193@KRBTEST.COM
I20260812 07:07:12.906070 11785 ts_tablet_manager.cc:1939] T c092866bf3c644a5a8493fe77ee56b26 P 2cc2fcf95cf3471fa4f1fb1269476cfb: Deleting consensus metadata
I20260812 07:07:12.906116 10898 catalog_manager.cc:4841] TS 15f3f16f1f3d480786d63525f7421f9a (127.11.192.195:42557): tablet c092866bf3c644a5a8493fe77ee56b26 (table test-table [id=1a7b24493bca4ac984bc3f1dd346bf16]) successfully deleted
I20260812 07:07:12.907671 10896 catalog_manager.cc:4841] TS 2cc2fcf95cf3471fa4f1fb1269476cfb (127.11.192.193:41397): tablet c092866bf3c644a5a8493fe77ee56b26 (table test-table [id=1a7b24493bca4ac984bc3f1dd346bf16]) successfully deleted
I20260812 07:07:12.913724 11739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:12.885002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53744 (local address 127.11.192.193:41397)
0812 07:07:12.885165 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:12.885170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:12.895243 (+ 10073us) server_negotiation.cc:407] Connection header received
0812 07:07:12.895692 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:12.895697 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:12.895782 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:12.895857 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:12.898180 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.899232 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.901144 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:12.901797 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:12.901891 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:12.907363 (+ 5472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:12.907379 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:12.907396 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:12.907443 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:12.910990 (+ 3547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:12.911618 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:12.911628 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:12.911631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:12.911785 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:12.912142 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:12.912148 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:12.912151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:12.912623 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:12.912927 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:12.913124 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 07:07:12.913343 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:07:12.914438 11241 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:13.002381 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 11039
I20260812 07:07:13.029353 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 11268
Aug 12 07:07:13 dist-test-slave-lgr5 krb5kdc[10845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.194@KRBTEST.COM
I20260812 07:07:13.050416 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:13.030146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40156 (local address 127.11.192.194:43975)
0812 07:07:13.030340 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:13.030346 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:13.030436 (+ 90us) server_negotiation.cc:407] Connection header received
0812 07:07:13.030647 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:13.030653 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:13.030732 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:13.030844 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:13.032507 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:13.033777 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:13.035647 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:13.036102 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:13.036236 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:13.044697 (+ 8461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:13.044711 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:13.044726 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:13.044767 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:13.047676 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:13.048353 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:13.048360 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:13.048363 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:13.048535 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:13.048900 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:13.048903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:13.048906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:13.049251 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:13.049593 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:13.049875 (+ 282us) server_negotiation.cc:299] Negotiation successful
0812 07:07:13.050094 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:07:13.051038 11464 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:13.172497 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 11268
I20260812 07:07:13.198442 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 11481
Aug 12 07:07:13 dist-test-slave-lgr5 krb5kdc[10845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.195@KRBTEST.COM
I20260812 07:07:13.212500 11743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:13.199177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35530 (local address 127.11.192.195:42557)
0812 07:07:13.199348 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:13.199354 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:13.199374 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:07:13.199566 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:13.199570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:13.199633 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:13.199716 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:13.200973 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:13.201705 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:13.202908 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:13.203295 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:13.203404 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:13.207387 (+ 3983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:13.207401 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:13.207416 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:13.207453 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:13.209912 (+ 2459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:13.210562 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:13.210569 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:13.210571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:13.210707 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:13.211047 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:13.211053 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:13.211055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:13.211385 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:13.211672 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:13.211962 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 07:07:13.212195 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:07:13.213125 11701 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:13.298897 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 11481
I20260812 07:07:13.321954 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 10875
I20260812 07:07:13.323100 10979 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:13.429976 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 10875
2026-08-12T07:07:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4363 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T07:07:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:13Z Disabled control of system clock
I20260812 07:07:13.512686 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:45817
--webserver_interface=127.11.192.254
--webserver_port=0
--builtin_ntp_servers=127.11.192.212:39163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:45817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:07:13.837306 11799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:13.837561 11799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:13.837683 11799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:13.844695 11799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:13.844794 11799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:13.844856 11799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:13.844933 11799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:13.844988 11799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:13.853822 11799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:39163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:45817
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:45817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:13.855024 11799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:13.856824 11799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:13.867750 11804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:13.868074 11807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:13.868156 11805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:13.868561 11799 server_base.cc:1034] running on GCE node
I20260812 07:07:13.869256 11799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:13.870471 11799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:13.871681 11799 hybrid_clock.cc:648] HybridClock initialized: now 1786518433871626 us; error 39 us; skew 500 ppm
I20260812 07:07:13.872008 11799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:07:13.874135 11799 webserver.cc:460] Webserver started at http://127.11.192.254:40019/ using document root <none> and password file <none>
I20260812 07:07:13.874747 11799 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:13.874845 11799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:13.875169 11799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:13.877908 11799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "2461ebb70eff46baa5d7e37986ee71c3"
format_stamp: "Formatted at 2026-08-12 07:07:13 on dist-test-slave-lgr5"
server_key: "ebf6d928d48cfc663aa44c27195054f9"
server_key_iv: "6856556076544eec77ce70298d790f39"
server_key_version: "encryptionkey@0"
I20260812 07:07:13.878682 11799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "2461ebb70eff46baa5d7e37986ee71c3"
format_stamp: "Formatted at 2026-08-12 07:07:13 on dist-test-slave-lgr5"
server_key: "ebf6d928d48cfc663aa44c27195054f9"
server_key_iv: "6856556076544eec77ce70298d790f39"
server_key_version: "encryptionkey@0"
I20260812 07:07:13.883399 11799 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 07:07:13.886567 11815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:13.887696 11799 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:07:13.887847 11799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "2461ebb70eff46baa5d7e37986ee71c3"
format_stamp: "Formatted at 2026-08-12 07:07:13 on dist-test-slave-lgr5"
server_key: "ebf6d928d48cfc663aa44c27195054f9"
server_key_iv: "6856556076544eec77ce70298d790f39"
server_key_version: "encryptionkey@0"
I20260812 07:07:13.888037 11799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:13.906244 11799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:13.906944 11799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:13.907193 11799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:13.923748 11799 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:45817
I20260812 07:07:13.923781 11904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:45817 every 8 connection(s)
I20260812 07:07:13.925009 11799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:13.928459 11905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:13.932652 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 11799
I20260812 07:07:13.932816 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:07:13.933245 12035 external_mini_cluster.cc:1428] Setting key c1dcf302fea6d64c108e660d337a7ed3
I20260812 07:07:13.938959 11905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3: Bootstrap starting.
I20260812 07:07:13.942649 11905 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:13.943825 11905 log.cc:824] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:13.946815 11905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3: No bootstrap required, opened a new log
I20260812 07:07:13.946849 11909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:13.934940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36662 (local address 127.11.192.254:45817)
0812 07:07:13.935856 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:13.935869 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:13.935912 (+ 43us) server_negotiation.cc:407] Connection header received
0812 07:07:13.937753 (+ 1841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:13.937776 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:13.938183 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:13.938478 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:13.939122 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:13.940451 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:13.942389 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:13.943064 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:13.943184 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:13.944574 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:13.944589 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:13.944612 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:13.945326 (+ 714us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:13.945408 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:13.945429 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:13.945520 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:13.945721 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 07:07:13.945982 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":318,"threads_started":1}
I20260812 07:07:13.954638 11905 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2461ebb70eff46baa5d7e37986ee71c3" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45817 } }
I20260812 07:07:13.954962 11905 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:13.955075 11905 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2461ebb70eff46baa5d7e37986ee71c3, State: Initialized, Role: FOLLOWER
I20260812 07:07:13.955574 11905 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2461ebb70eff46baa5d7e37986ee71c3" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45817 } }
I20260812 07:07:13.955780 11905 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:13.955906 11905 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:13.956053 11905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:13.958261 11905 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2461ebb70eff46baa5d7e37986ee71c3" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45817 } }
I20260812 07:07:13.958761 11905 leader_election.cc:302] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2461ebb70eff46baa5d7e37986ee71c3; no voters:
I20260812 07:07:13.959323 11905 leader_election.cc:288] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:13.959425 11912 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:13.960060 11912 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3 [term 1 LEADER]: Becoming Leader. State: Replica: 2461ebb70eff46baa5d7e37986ee71c3, State: Running, Role: LEADER
I20260812 07:07:13.960568 11912 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2461ebb70eff46baa5d7e37986ee71c3" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45817 } }
I20260812 07:07:13.961189 11905 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:13.963042 11915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2461ebb70eff46baa5d7e37986ee71c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2461ebb70eff46baa5d7e37986ee71c3" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45817 } } }
I20260812 07:07:13.963065 11916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2461ebb70eff46baa5d7e37986ee71c3. Latest consensus state: current_term: 1 leader_uuid: "2461ebb70eff46baa5d7e37986ee71c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2461ebb70eff46baa5d7e37986ee71c3" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 45817 } } }
I20260812 07:07:13.963477 11916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:13.963486 11915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:13.969133 11922 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:13.978884 11922 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:13.988523 11922 catalog_manager.cc:1347] Generated new cluster ID: cf0bd0aefef1485ea69ccb59600ae3bb
I20260812 07:07:13.988633 11922 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:14.026489 11922 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:14.027724 11922 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:14.036476 11922 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3: Generated new TSK 0
I20260812 07:07:14.037317 11922 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:07:14.054947 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:45817
--builtin_ntp_servers=127.11.192.212:39163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:07:14.414306 11943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:14.414561 11943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:14.414685 11943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:14.421686 11943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:14.421775 11943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:14.421926 11943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:07:14.430577 11943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:39163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:45817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:14.431789 11943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:14.433535 11943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:14.445510 11954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:14.445540 11959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:14.445798 11956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:14.446503 11943 server_base.cc:1034] running on GCE node
I20260812 07:07:14.447067 11943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:14.448009 11943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:14.449291 11943 hybrid_clock.cc:648] HybridClock initialized: now 1786518434449229 us; error 49 us; skew 500 ppm
I20260812 07:07:14.449561 11943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:07:14.451627 11943 webserver.cc:460] Webserver started at http://127.11.192.193:45365/ using document root <none> and password file <none>
I20260812 07:07:14.452292 11943 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:14.452384 11943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:14.452641 11943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:14.455315 11943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "a1d45e34311547e79e95f278a049ae19"
format_stamp: "Formatted at 2026-08-12 07:07:14 on dist-test-slave-lgr5"
server_key: "8f5c931907c9d60c92f60f29bc83cb27"
server_key_iv: "604bb31c5981afd674db4e4861408bf2"
server_key_version: "encryptionkey@0"
I20260812 07:07:14.456073 11943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "a1d45e34311547e79e95f278a049ae19"
format_stamp: "Formatted at 2026-08-12 07:07:14 on dist-test-slave-lgr5"
server_key: "8f5c931907c9d60c92f60f29bc83cb27"
server_key_iv: "604bb31c5981afd674db4e4861408bf2"
server_key_version: "encryptionkey@0"
I20260812 07:07:14.460772 11943 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:07:14.464341 11971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:14.465531 11943 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:07:14.465680 11943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "a1d45e34311547e79e95f278a049ae19"
format_stamp: "Formatted at 2026-08-12 07:07:14 on dist-test-slave-lgr5"
server_key: "8f5c931907c9d60c92f60f29bc83cb27"
server_key_iv: "604bb31c5981afd674db4e4861408bf2"
server_key_version: "encryptionkey@0"
I20260812 07:07:14.465875 11943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:14.491966 11943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:14.492729 11943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:14.493019 11943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:14.493913 11943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:14.495260 11943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:14.495352 11943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:14.495471 11943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:14.495543 11943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:14.525206 11943 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:36367
I20260812 07:07:14.525270 12155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:36367 every 8 connection(s)
I20260812 07:07:14.526423 11943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:07:14.529425 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 11943
I20260812 07:07:14.529603 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:07:14.530205 12035 external_mini_cluster.cc:1428] Setting key a576b9332de3fc26b8dc250396a9e10d
I20260812 07:07:14.532903 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:45817
--builtin_ntp_servers=127.11.192.212:39163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:07:14.538856 12164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:14.528861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:35621 (local address 127.11.192.254:45817)
0812 07:07:14.529293 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:14.529297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:14.530560 (+ 1263us) server_negotiation.cc:407] Connection header received
0812 07:07:14.532494 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:14.532500 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:14.532578 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:14.532668 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:14.533725 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:14.534462 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:14.535936 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:14.536414 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:14.536523 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:14.537863 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:14.537869 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:14.537873 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:14.537894 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:14.537940 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:14.537955 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:14.538047 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:14.538222 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 07:07:14.538311 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":227,"threads_started":1}
I20260812 07:07:14.539525 12165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:14.529434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45817 (local address 127.11.192.193:35621)
0812 07:07:14.530414 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:14.530450 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:14.532247 (+ 1797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:14.532891 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:14.532899 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:14.532993 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:14.533567 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:14.533584 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:14.534608 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:14.534612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:14.535799 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:14.535811 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:14.536623 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:14.536630 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:14.536839 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:14.537532 (+ 693us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:07:14.537536 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:14.537588 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:07:14.537671 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:14.538046 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:14.538054 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:14.538060 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 07:07:14.538195 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 07:07:14.538506 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":292,"threads_started":1}
I20260812 07:07:14.540884 12156 heartbeater.cc:347] Connected to a master server at 127.11.192.254:45817
I20260812 07:07:14.541185 12156 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:14.541971 12156 heartbeater.cc:511] Master 127.11.192.254:45817 requested a full tablet report, sending...
I20260812 07:07:14.543998 11842 ts_manager.cc:194] Registered new tserver with Master: a1d45e34311547e79e95f278a049ae19 (127.11.192.193:36367)
I20260812 07:07:14.545712 11842 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.192.193:35621
W20260812 07:07:14.861526 12169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:14.861781 12169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:14.861898 12169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:14.868922 12169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:14.869014 12169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:14.869160 12169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:07:14.878072 12169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:39163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:45817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:14.879168 12169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:14.880926 12169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:14.893388 12184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:14.893496 12181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:14.893452 12182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:14.894276 12169 server_base.cc:1034] running on GCE node
I20260812 07:07:14.894798 12169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:14.895612 12169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:14.896799 12169 hybrid_clock.cc:648] HybridClock initialized: now 1786518434896744 us; error 48 us; skew 500 ppm
I20260812 07:07:14.897085 12169 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:07:14.899143 12169 webserver.cc:460] Webserver started at http://127.11.192.194:44019/ using document root <none> and password file <none>
I20260812 07:07:14.899772 12169 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:14.899863 12169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:14.900197 12169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:14.902949 12169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "f6a900be3651454a8a12be45daf1abf8"
format_stamp: "Formatted at 2026-08-12 07:07:14 on dist-test-slave-lgr5"
server_key: "36f539f4b49235efd65f8ff7bb134e8e"
server_key_iv: "ac36eb96e62a95148e15c132bd80c9bb"
server_key_version: "encryptionkey@0"
I20260812 07:07:14.903760 12169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "f6a900be3651454a8a12be45daf1abf8"
format_stamp: "Formatted at 2026-08-12 07:07:14 on dist-test-slave-lgr5"
server_key: "36f539f4b49235efd65f8ff7bb134e8e"
server_key_iv: "ac36eb96e62a95148e15c132bd80c9bb"
server_key_version: "encryptionkey@0"
I20260812 07:07:14.908514 12169 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 07:07:14.911775 12194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:14.913024 12169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:07:14.913170 12169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "f6a900be3651454a8a12be45daf1abf8"
format_stamp: "Formatted at 2026-08-12 07:07:14 on dist-test-slave-lgr5"
server_key: "36f539f4b49235efd65f8ff7bb134e8e"
server_key_iv: "ac36eb96e62a95148e15c132bd80c9bb"
server_key_version: "encryptionkey@0"
I20260812 07:07:14.913355 12169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:14.929646 12169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:14.930313 12169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:14.930581 12169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:14.931481 12169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:14.932871 12169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:14.932963 12169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:14.933063 12169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:14.933131 12169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:14.964730 12169 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:34961
I20260812 07:07:14.964793 12374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:34961 every 8 connection(s)
I20260812 07:07:14.965957 12169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:07:14.973878 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 12169
I20260812 07:07:14.974056 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:07:14.974519 12035 external_mini_cluster.cc:1428] Setting key 1cdf13de9eb81fc5fc75a5dd913964a4
I20260812 07:07:14.977217 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:45817
--builtin_ntp_servers=127.11.192.212:39163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:07:14.978084 12164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:14.968361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:39033 (local address 127.11.192.254:45817)
0812 07:07:14.968539 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:14.968544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:14.970018 (+ 1474us) server_negotiation.cc:407] Connection header received
0812 07:07:14.971881 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:14.971887 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:14.971974 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:14.972063 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:14.973135 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:14.973851 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:14.975438 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:14.975867 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:14.975980 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:14.977250 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:14.977255 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:14.977259 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:14.977276 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:14.977308 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:14.977354 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:14.977431 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:14.977702 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 07:07:14.977795 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 07:07:14.978931 12377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:14.968873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45817 (local address 127.11.192.194:39033)
0812 07:07:14.969838 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:14.969896 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:14.971617 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:14.972326 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:14.972335 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:14.972418 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:14.972973 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:14.972990 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:14.973995 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:14.974001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:14.975291 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:14.975305 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:14.976121 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:14.976135 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:14.976347 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:14.976964 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:07:14.976967 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:14.977003 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:07:14.977074 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:14.977482 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:14.977491 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:14.977497 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 07:07:14.977627 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 07:07:14.977907 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":269,"threads_started":1}
I20260812 07:07:14.980345 12375 heartbeater.cc:347] Connected to a master server at 127.11.192.254:45817
I20260812 07:07:14.980619 12375 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:14.981329 12375 heartbeater.cc:511] Master 127.11.192.254:45817 requested a full tablet report, sending...
I20260812 07:07:14.982744 11842 ts_manager.cc:194] Registered new tserver with Master: f6a900be3651454a8a12be45daf1abf8 (127.11.192.194:34961)
I20260812 07:07:14.983680 11842 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.192.194:39033
W20260812 07:07:15.301982 12378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:15.302248 12378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:15.302386 12378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:15.309345 12378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:15.309446 12378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:15.309592 12378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:07:15.318271 12378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:39163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:45817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:15.319398 12378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:15.321135 12378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:15.332994 12387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:15.333011 12388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:15.333016 12392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:15.334136 12378 server_base.cc:1034] running on GCE node
I20260812 07:07:15.334630 12378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:15.335505 12378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:15.336700 12378 hybrid_clock.cc:648] HybridClock initialized: now 1786518435336649 us; error 42 us; skew 500 ppm
I20260812 07:07:15.336997 12378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:07:15.339020 12378 webserver.cc:460] Webserver started at http://127.11.192.195:34283/ using document root <none> and password file <none>
I20260812 07:07:15.339622 12378 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:15.339718 12378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:15.340013 12378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:15.342864 12378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "44cfa5a18eac46c6865534cd7163d872"
format_stamp: "Formatted at 2026-08-12 07:07:15 on dist-test-slave-lgr5"
server_key: "90e6f7a73833d37e3dd4edcbed5223b7"
server_key_iv: "6383ca52ad8963d10f329fcc075ab8a3"
server_key_version: "encryptionkey@0"
I20260812 07:07:15.343605 12378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "44cfa5a18eac46c6865534cd7163d872"
format_stamp: "Formatted at 2026-08-12 07:07:15 on dist-test-slave-lgr5"
server_key: "90e6f7a73833d37e3dd4edcbed5223b7"
server_key_iv: "6383ca52ad8963d10f329fcc075ab8a3"
server_key_version: "encryptionkey@0"
I20260812 07:07:15.348352 12378 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260812 07:07:15.351461 12401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:15.352658 12378 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:07:15.352800 12378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "44cfa5a18eac46c6865534cd7163d872"
format_stamp: "Formatted at 2026-08-12 07:07:15 on dist-test-slave-lgr5"
server_key: "90e6f7a73833d37e3dd4edcbed5223b7"
server_key_iv: "6383ca52ad8963d10f329fcc075ab8a3"
server_key_version: "encryptionkey@0"
I20260812 07:07:15.352991 12378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:15.376768 12378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:15.377436 12378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:15.377705 12378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:15.378542 12378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:15.379901 12378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:15.380000 12378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:15.380118 12378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:15.380194 12378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260812 07:07:15.412578 12378 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:37793
I20260812 07:07:15.412654 12587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:37793 every 8 connection(s)
I20260812 07:07:15.413789 12378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:07:15.415654 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 12378
I20260812 07:07:15.415854 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:07:15.416486 12035 external_mini_cluster.cc:1428] Setting key baccdd8d1219f95417fec7e1c778099d
I20260812 07:07:15.425304 12164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.416231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:33061 (local address 127.11.192.254:45817)
0812 07:07:15.416417 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:15.416422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:15.418060 (+ 1638us) server_negotiation.cc:407] Connection header received
0812 07:07:15.419784 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:15.419790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:15.419873 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:15.419959 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:15.420996 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.421703 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.423133 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.423525 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.423610 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:15.424730 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:15.424735 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:15.424738 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:15.424754 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:15.424781 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:15.424792 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:15.424856 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:15.425013 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 07:07:15.425084 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 07:07:15.426142 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.416892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:45817 (local address 127.11.192.195:33061)
0812 07:07:15.417908 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:15.417951 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:15.419558 (+ 1607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:15.420203 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:15.420211 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:15.420297 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:15.420847 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:15.420863 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.421843 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.421847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:15.423026 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:15.423038 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.423687 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.423691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:15.423878 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:15.424487 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:07:15.424490 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:15.424526 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:07:15.424598 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:15.424859 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:15.424865 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:15.424870 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 07:07:15.424985 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 07:07:15.425238 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":311,"threads_started":1}
I20260812 07:07:15.427255 12588 heartbeater.cc:347] Connected to a master server at 127.11.192.254:45817
I20260812 07:07:15.427484 12588 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:15.428154 12588 heartbeater.cc:511] Master 127.11.192.254:45817 requested a full tablet report, sending...
I20260812 07:07:15.429394 11842 ts_manager.cc:194] Registered new tserver with Master: 44cfa5a18eac46c6865534cd7163d872 (127.11.192.195:37793)
I20260812 07:07:15.430378 11842 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.192.195:33061
I20260812 07:07:15.431134 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:07:15.440385 12164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.434970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36664 (local address 127.11.192.254:45817)
0812 07:07:15.435142 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:15.435147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:15.435553 (+ 406us) server_negotiation.cc:407] Connection header received
0812 07:07:15.435701 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:15.435705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:15.435769 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:15.435845 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:15.436360 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.436988 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.438355 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.438751 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.438853 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:15.439758 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:15.439763 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:15.439766 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:15.439780 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:15.439807 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:15.439817 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:15.439885 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:15.440025 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 07:07:15.440121 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 07:07:15.443658 11840 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:07:15.450213 11840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:07:15.471607 12610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.461260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:37793 (local address 127.0.0.1:41006)
0812 07:07:15.462103 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:15.462153 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:15.462322 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:15.463529 (+ 1207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:15.463536 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:15.463567 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:15.463993 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:15.464008 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.466286 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.466292 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:15.468707 (+ 2415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:15.468725 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.470674 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.470682 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:15.470909 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:15.470966 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 07:07:15.471068 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 07:07:15.471251 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":379,"threads_started":1}
I20260812 07:07:15.473058 12611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.461432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41006 (local address 127.11.192.195:37793)
0812 07:07:15.462917 (+ 1485us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:15.462923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:15.462949 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:07:15.463038 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:15.463044 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:15.463275 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:15.464241 (+ 966us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:15.464531 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.466125 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.468977 (+ 2852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.470424 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.472267 (+ 1843us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:15.472477 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:15.472556 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 07:07:15.472717 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1341,"thread_start_us":1203,"threads_started":1}
I20260812 07:07:15.479748 12610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.462403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:34961 (local address 127.0.0.1:49444)
0812 07:07:15.471835 (+ 9432us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:15.471854 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:15.471951 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:15.472557 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:15.472561 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:15.472586 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:15.472746 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:15.472756 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.475090 (+ 2334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.475096 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:15.477127 (+ 2031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:15.477159 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.478992 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.479000 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:15.479240 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:15.479276 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 07:07:15.479338 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 07:07:15.479473 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9320,"thread_start_us":15618,"threads_started":1}
I20260812 07:07:15.480605 12613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.462784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49444 (local address 127.11.192.194:34961)
0812 07:07:15.463363 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:15.463370 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:15.471931 (+ 8561us) server_negotiation.cc:407] Connection header received
0812 07:07:15.472110 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:15.472119 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:15.472342 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:15.472914 (+ 572us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:15.473189 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.474874 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.477354 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.478799 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.479937 (+ 1138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:15.480119 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:15.480176 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 07:07:15.480289 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":274,"threads_started":1}
I20260812 07:07:15.481901 12480 tablet_service.cc:1467] Processing CreateTablet for tablet 6a1a2ddb2ab14ef39c498a061ee449d5 (DEFAULT_TABLE table=test-table [id=4a7ba75eb163478d8398a12fb6ad27bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:15.483444 12480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a1a2ddb2ab14ef39c498a061ee449d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:15.484027 12276 tablet_service.cc:1467] Processing CreateTablet for tablet 6a1a2ddb2ab14ef39c498a061ee449d5 (DEFAULT_TABLE table=test-table [id=4a7ba75eb163478d8398a12fb6ad27bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:15.485097 12612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.463521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:36367 (local address 127.0.0.1:51534)
0812 07:07:15.476805 (+ 13284us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:15.476821 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:15.476930 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:15.477845 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:15.477849 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:15.477872 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:15.478283 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:15.478296 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.480405 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.480411 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:15.481907 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:15.481923 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.484297 (+ 2374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.484304 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:15.484471 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:15.484502 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 07:07:15.484700 (+ 198us) client_negotiation.cc:240] Negotiation successful
0812 07:07:15.484819 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13139,"thread_start_us":16066,"threads_started":1}
I20260812 07:07:15.485216 12616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.463746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51534 (local address 127.11.192.193:36367)
0812 07:07:15.477408 (+ 13662us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:15.477415 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:15.477437 (+ 22us) server_negotiation.cc:407] Connection header received
0812 07:07:15.477508 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:15.477513 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:15.477675 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:15.477827 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:15.478535 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.480269 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.482124 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.484109 (+ 1985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.484278 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:15.484462 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:15.484750 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 07:07:15.484914 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13504,"thread_start_us":13405,"threads_started":1}
I20260812 07:07:15.485350 12276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a1a2ddb2ab14ef39c498a061ee449d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:15.488744 12053 tablet_service.cc:1467] Processing CreateTablet for tablet 6a1a2ddb2ab14ef39c498a061ee449d5 (DEFAULT_TABLE table=test-table [id=4a7ba75eb163478d8398a12fb6ad27bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:15.490041 12053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a1a2ddb2ab14ef39c498a061ee449d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:15.495075 12618 tablet_bootstrap.cc:492] T 6a1a2ddb2ab14ef39c498a061ee449d5 P 44cfa5a18eac46c6865534cd7163d872: Bootstrap starting.
I20260812 07:07:15.498190 12618 tablet_bootstrap.cc:654] T 6a1a2ddb2ab14ef39c498a061ee449d5 P 44cfa5a18eac46c6865534cd7163d872: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:15.499439 12622 tablet_bootstrap.cc:492] T 6a1a2ddb2ab14ef39c498a061ee449d5 P f6a900be3651454a8a12be45daf1abf8: Bootstrap starting.
I20260812 07:07:15.499890 12618 log.cc:824] T 6a1a2ddb2ab14ef39c498a061ee449d5 P 44cfa5a18eac46c6865534cd7163d872: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:15.502671 12622 tablet_bootstrap.cc:654] T 6a1a2ddb2ab14ef39c498a061ee449d5 P f6a900be3651454a8a12be45daf1abf8: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:15.502936 12618 tablet_bootstrap.cc:492] T 6a1a2ddb2ab14ef39c498a061ee449d5 P 44cfa5a18eac46c6865534cd7163d872: No bootstrap required, opened a new log
I20260812 07:07:15.502969 12631 tablet_bootstrap.cc:492] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19: Bootstrap starting.
I20260812 07:07:15.503211 12618 ts_tablet_manager.cc:1397] T 6a1a2ddb2ab14ef39c498a061ee449d5 P 44cfa5a18eac46c6865534cd7163d872: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 07:07:15.503911 12622 log.cc:824] T 6a1a2ddb2ab14ef39c498a061ee449d5 P f6a900be3651454a8a12be45daf1abf8: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:15.505867 12631 tablet_bootstrap.cc:654] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:15.506701 12631 log.cc:824] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:15.506796 12622 tablet_bootstrap.cc:492] T 6a1a2ddb2ab14ef39c498a061ee449d5 P f6a900be3651454a8a12be45daf1abf8: No bootstrap required, opened a new log
I20260812 07:07:15.507067 12622 ts_tablet_manager.cc:1397] T 6a1a2ddb2ab14ef39c498a061ee449d5 P f6a900be3651454a8a12be45daf1abf8: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 07:07:15.509218 12631 tablet_bootstrap.cc:492] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19: No bootstrap required, opened a new log
I20260812 07:07:15.509440 12631 ts_tablet_manager.cc:1397] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260812 07:07:15.512830 12618 raft_consensus.cc:348] T 6a1a2ddb2ab14ef39c498a061ee449d5 P 44cfa5a18eac46c6865534cd7163d872 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44cfa5a18eac46c6865534cd7163d872" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 37793 } } peers { permanent_uuid: "f6a900be3651454a8a12be45daf1abf8" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34961 } } peers { permanent_uuid: "a1d45e34311547e79e95f278a049ae19" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 36367 } }
I20260812 07:07:15.513247 12618 raft_consensus.cc:374] T 6a1a2ddb2ab14ef39c498a061ee449d5 P 44cfa5a18eac46c6865534cd7163d872 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:15.513342 12618 raft_consensus.cc:729] T 6a1a2ddb2ab14ef39c498a061ee449d5 P 44cfa5a18eac46c6865534cd7163d872 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44cfa5a18eac46c6865534cd7163d872, State: Initialized, Role: FOLLOWER
I20260812 07:07:15.513952 12618 consensus_queue.cc:260] T 6a1a2ddb2ab14ef39c498a061ee449d5 P 44cfa5a18eac46c6865534cd7163d872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44cfa5a18eac46c6865534cd7163d872" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 37793 } } peers { permanent_uuid: "f6a900be3651454a8a12be45daf1abf8" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34961 } } peers { permanent_uuid: "a1d45e34311547e79e95f278a049ae19" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 36367 } }
I20260812 07:07:15.515007 12588 heartbeater.cc:503] Master 127.11.192.254:45817 was elected leader, sending a full tablet report...
I20260812 07:07:15.515618 12618 ts_tablet_manager.cc:1428] T 6a1a2ddb2ab14ef39c498a061ee449d5 P 44cfa5a18eac46c6865534cd7163d872: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260812 07:07:15.516453 12631 raft_consensus.cc:348] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44cfa5a18eac46c6865534cd7163d872" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 37793 } } peers { permanent_uuid: "f6a900be3651454a8a12be45daf1abf8" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34961 } } peers { permanent_uuid: "a1d45e34311547e79e95f278a049ae19" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 36367 } }
I20260812 07:07:15.516834 12631 raft_consensus.cc:374] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:15.516930 12631 raft_consensus.cc:729] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1d45e34311547e79e95f278a049ae19, State: Initialized, Role: FOLLOWER
I20260812 07:07:15.517437 12631 consensus_queue.cc:260] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44cfa5a18eac46c6865534cd7163d872" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 37793 } } peers { permanent_uuid: "f6a900be3651454a8a12be45daf1abf8" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34961 } } peers { permanent_uuid: "a1d45e34311547e79e95f278a049ae19" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 36367 } }
I20260812 07:07:15.518299 12622 raft_consensus.cc:348] T 6a1a2ddb2ab14ef39c498a061ee449d5 P f6a900be3651454a8a12be45daf1abf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44cfa5a18eac46c6865534cd7163d872" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 37793 } } peers { permanent_uuid: "f6a900be3651454a8a12be45daf1abf8" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34961 } } peers { permanent_uuid: "a1d45e34311547e79e95f278a049ae19" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 36367 } }
I20260812 07:07:15.518738 12622 raft_consensus.cc:374] T 6a1a2ddb2ab14ef39c498a061ee449d5 P f6a900be3651454a8a12be45daf1abf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:15.518847 12622 raft_consensus.cc:729] T 6a1a2ddb2ab14ef39c498a061ee449d5 P f6a900be3651454a8a12be45daf1abf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6a900be3651454a8a12be45daf1abf8, State: Initialized, Role: FOLLOWER
I20260812 07:07:15.519687 12631 ts_tablet_manager.cc:1428] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
I20260812 07:07:15.519515 12622 consensus_queue.cc:260] T 6a1a2ddb2ab14ef39c498a061ee449d5 P f6a900be3651454a8a12be45daf1abf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44cfa5a18eac46c6865534cd7163d872" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 37793 } } peers { permanent_uuid: "f6a900be3651454a8a12be45daf1abf8" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34961 } } peers { permanent_uuid: "a1d45e34311547e79e95f278a049ae19" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 36367 } }
I20260812 07:07:15.520066 12156 heartbeater.cc:503] Master 127.11.192.254:45817 was elected leader, sending a full tablet report...
I20260812 07:07:15.522939 12375 heartbeater.cc:503] Master 127.11.192.254:45817 was elected leader, sending a full tablet report...
I20260812 07:07:15.523561 12622 ts_tablet_manager.cc:1428] T 6a1a2ddb2ab14ef39c498a061ee449d5 P f6a900be3651454a8a12be45daf1abf8: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
W20260812 07:07:15.528584 12158 tablet.cc:2369] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:07:15.528725 12637 raft_consensus.cc:482] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:07:15.528986 12637 raft_consensus.cc:504] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44cfa5a18eac46c6865534cd7163d872" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 37793 } } peers { permanent_uuid: "f6a900be3651454a8a12be45daf1abf8" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34961 } } peers { permanent_uuid: "a1d45e34311547e79e95f278a049ae19" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 36367 } }
I20260812 07:07:15.530421 12637 leader_election.cc:288] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44cfa5a18eac46c6865534cd7163d872 (127.11.192.195:37793), f6a900be3651454a8a12be45daf1abf8 (127.11.192.194:34961)
I20260812 07:07:15.536880 12611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.530825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:56131 (local address 127.11.192.195:37793)
0812 07:07:15.530981 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:15.530986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:15.531522 (+ 536us) server_negotiation.cc:407] Connection header received
0812 07:07:15.531662 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:15.531667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:15.531738 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:15.531797 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:15.532470 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.533308 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.535080 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.535834 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.535975 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:15.536054 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:15.536520 (+ 466us) server_negotiation.cc:299] Negotiation successful
0812 07:07:15.536660 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 07:07:15.536888 12639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.530800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:37793 (local address 127.11.192.193:56131)
0812 07:07:15.531347 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:15.531364 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:15.531494 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:15.531956 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:15.531959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:15.531980 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:15.532339 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:15.532352 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.533431 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.533436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:15.534886 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:15.534900 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.536011 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.536018 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:15.536275 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:15.536310 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 07:07:15.536471 (+ 161us) client_negotiation.cc:240] Negotiation successful
0812 07:07:15.536613 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":294,"threads_started":1}
I20260812 07:07:15.538051 12514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a1a2ddb2ab14ef39c498a061ee449d5" candidate_uuid: "a1d45e34311547e79e95f278a049ae19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44cfa5a18eac46c6865534cd7163d872" is_pre_election: true
I20260812 07:07:15.538578 12514 raft_consensus.cc:2454] T 6a1a2ddb2ab14ef39c498a061ee449d5 P 44cfa5a18eac46c6865534cd7163d872 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1d45e34311547e79e95f278a049ae19 in term 0.
I20260812 07:07:15.539198 11976 leader_election.cc:302] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44cfa5a18eac46c6865534cd7163d872, a1d45e34311547e79e95f278a049ae19; no voters:
I20260812 07:07:15.539547 12637 raft_consensus.cc:2781] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:07:15.539660 12637 raft_consensus.cc:482] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:07:15.539755 12637 raft_consensus.cc:3037] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:15.542035 12637 raft_consensus.cc:504] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44cfa5a18eac46c6865534cd7163d872" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 37793 } } peers { permanent_uuid: "f6a900be3651454a8a12be45daf1abf8" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34961 } } peers { permanent_uuid: "a1d45e34311547e79e95f278a049ae19" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 36367 } }
I20260812 07:07:15.542675 12637 leader_election.cc:288] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19 [CANDIDATE]: Term 1 election: Requested vote from peers 44cfa5a18eac46c6865534cd7163d872 (127.11.192.195:37793), f6a900be3651454a8a12be45daf1abf8 (127.11.192.194:34961)
I20260812 07:07:15.543118 12514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a1a2ddb2ab14ef39c498a061ee449d5" candidate_uuid: "a1d45e34311547e79e95f278a049ae19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44cfa5a18eac46c6865534cd7163d872"
I20260812 07:07:15.543272 12514 raft_consensus.cc:3037] T 6a1a2ddb2ab14ef39c498a061ee449d5 P 44cfa5a18eac46c6865534cd7163d872 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:15.543850 12613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.531641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:40203 (local address 127.11.192.194:34961)
0812 07:07:15.531791 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:15.531797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:15.537432 (+ 5635us) server_negotiation.cc:407] Connection header received
0812 07:07:15.537732 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:15.537737 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:15.537799 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:15.537869 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:15.538402 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.539369 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.541751 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.542864 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.543004 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:15.543077 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:15.543492 (+ 415us) server_negotiation.cc:299] Negotiation successful
0812 07:07:15.543621 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:07:15.543874 12642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.531508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:34961 (local address 127.11.192.193:40203)
0812 07:07:15.536594 (+ 5086us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:15.537357 (+ 763us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:15.537527 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:15.537898 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:15.537903 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:15.537926 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:15.538246 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:15.538255 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.540157 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.540161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:15.541570 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:15.541584 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.543186 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.543191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:15.543345 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:15.543378 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:07:15.543461 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 07:07:15.543572 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4968,"thread_start_us":4781,"threads_started":1}
I20260812 07:07:15.544922 12309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a1a2ddb2ab14ef39c498a061ee449d5" candidate_uuid: "a1d45e34311547e79e95f278a049ae19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6a900be3651454a8a12be45daf1abf8" is_pre_election: true
I20260812 07:07:15.544955 12308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a1a2ddb2ab14ef39c498a061ee449d5" candidate_uuid: "a1d45e34311547e79e95f278a049ae19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6a900be3651454a8a12be45daf1abf8"
I20260812 07:07:15.545365 12309 raft_consensus.cc:2454] T 6a1a2ddb2ab14ef39c498a061ee449d5 P f6a900be3651454a8a12be45daf1abf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1d45e34311547e79e95f278a049ae19 in term 0.
I20260812 07:07:15.545873 12514 raft_consensus.cc:2454] T 6a1a2ddb2ab14ef39c498a061ee449d5 P 44cfa5a18eac46c6865534cd7163d872 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1d45e34311547e79e95f278a049ae19 in term 1.
I20260812 07:07:15.546243 11976 leader_election.cc:302] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 44cfa5a18eac46c6865534cd7163d872, a1d45e34311547e79e95f278a049ae19; no voters: f6a900be3651454a8a12be45daf1abf8
I20260812 07:07:15.546470 12637 raft_consensus.cc:2781] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:15.546828 12637 raft_consensus.cc:686] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19 [term 1 LEADER]: Becoming Leader. State: Replica: a1d45e34311547e79e95f278a049ae19, State: Running, Role: LEADER
I20260812 07:07:15.547252 12637 consensus_queue.cc:237] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44cfa5a18eac46c6865534cd7163d872" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 37793 } } peers { permanent_uuid: "f6a900be3651454a8a12be45daf1abf8" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34961 } } peers { permanent_uuid: "a1d45e34311547e79e95f278a049ae19" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 36367 } }
I20260812 07:07:15.551390 11839 catalog_manager.cc:5495] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1d45e34311547e79e95f278a049ae19 (127.11.192.193). New cstate: current_term: 1 leader_uuid: "a1d45e34311547e79e95f278a049ae19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44cfa5a18eac46c6865534cd7163d872" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 37793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6a900be3651454a8a12be45daf1abf8" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1d45e34311547e79e95f278a049ae19" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 36367 } health_report { overall_health: HEALTHY } } }
I20260812 07:07:15.567127 12616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.562073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51542 (local address 127.11.192.193:36367)
0812 07:07:15.562249 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:15.562255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:15.562272 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:07:15.562447 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:15.562451 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:15.562534 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:15.562653 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:07:15.562998 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.563664 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.565133 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.565539 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.565642 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:15.566125 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:07:15.566582 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:07:15.566705 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:15.566810 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 07:07:15.566885 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 07:07:15.574620 12308 raft_consensus.cc:3037] T 6a1a2ddb2ab14ef39c498a061ee449d5 P f6a900be3651454a8a12be45daf1abf8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:15.574774 12514 raft_consensus.cc:1261] T 6a1a2ddb2ab14ef39c498a061ee449d5 P 44cfa5a18eac46c6865534cd7163d872 [term 1 FOLLOWER]: Refusing update from remote peer a1d45e34311547e79e95f278a049ae19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:07:15.575556 12644 consensus_queue.cc:1035] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44cfa5a18eac46c6865534cd7163d872" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 37793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:07:15.577256 12308 raft_consensus.cc:1261] T 6a1a2ddb2ab14ef39c498a061ee449d5 P f6a900be3651454a8a12be45daf1abf8 [term 1 FOLLOWER]: Refusing update from remote peer a1d45e34311547e79e95f278a049ae19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:07:15.577925 12644 consensus_queue.cc:1035] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6a900be3651454a8a12be45daf1abf8" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:07:15.588302 12654 mvcc.cc:205] Tried to move back new op lower bound from 7317579512105713664 to 7317579512005246976. Current Snapshot: MvccSnapshot[applied={T|T < 7317579512105713664}]
I20260812 07:07:15.593606 12649 mvcc.cc:205] Tried to move back new op lower bound from 7317579512105713664 to 7317579512005246976. Current Snapshot: MvccSnapshot[applied={T|T < 7317579512105713664}]
I20260812 07:07:15.600541 11839 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:07:15.600772 11839 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:07:15.603997 11839 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2461ebb70eff46baa5d7e37986ee71c3: Sending DeleteTablet for 3 replicas of tablet 6a1a2ddb2ab14ef39c498a061ee449d5
I20260812 07:07:15.605159 12480 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a1a2ddb2ab14ef39c498a061ee449d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:15 UTC) from {username='slave'} at 127.0.0.1:41006
I20260812 07:07:15.605159 12276 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a1a2ddb2ab14ef39c498a061ee449d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:15 UTC) from {username='slave'} at 127.0.0.1:49444
I20260812 07:07:15.605194 12053 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a1a2ddb2ab14ef39c498a061ee449d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:15 UTC) from {username='slave'} at 127.0.0.1:51534
I20260812 07:07:15.611617 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 11943
I20260812 07:07:15.616302 12671 tablet_replica.cc:330] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19: stopping tablet replica
I20260812 07:07:15.616839 12671 raft_consensus.cc:2229] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:07:15.617511 12671 raft_consensus.cc:2258] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:15.618317 12669 tablet_replica.cc:330] T 6a1a2ddb2ab14ef39c498a061ee449d5 P 44cfa5a18eac46c6865534cd7163d872: stopping tablet replica
I20260812 07:07:15.618563 12670 tablet_replica.cc:330] T 6a1a2ddb2ab14ef39c498a061ee449d5 P f6a900be3651454a8a12be45daf1abf8: stopping tablet replica
I20260812 07:07:15.618717 12669 raft_consensus.cc:2229] T 6a1a2ddb2ab14ef39c498a061ee449d5 P 44cfa5a18eac46c6865534cd7163d872 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:07:15.618893 12670 raft_consensus.cc:2229] T 6a1a2ddb2ab14ef39c498a061ee449d5 P f6a900be3651454a8a12be45daf1abf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:07:15.619407 12670 raft_consensus.cc:2258] T 6a1a2ddb2ab14ef39c498a061ee449d5 P f6a900be3651454a8a12be45daf1abf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:15.619607 12669 raft_consensus.cc:2258] T 6a1a2ddb2ab14ef39c498a061ee449d5 P 44cfa5a18eac46c6865534cd7163d872 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:15.620690 12671 ts_tablet_manager.cc:1905] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:15.621551 12669 ts_tablet_manager.cc:1905] T 6a1a2ddb2ab14ef39c498a061ee449d5 P 44cfa5a18eac46c6865534cd7163d872: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:15.621539 12670 ts_tablet_manager.cc:1905] T 6a1a2ddb2ab14ef39c498a061ee449d5 P f6a900be3651454a8a12be45daf1abf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:15.625417 12616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.612720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51552 (local address 127.11.192.193:36367)
0812 07:07:15.612895 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:15.612902 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:15.616498 (+ 3596us) server_negotiation.cc:407] Connection header received
0812 07:07:15.616814 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:15.616820 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:15.616914 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:15.616999 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:15.617791 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.618859 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.621631 (+ 2772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.622268 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.622371 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:15.623749 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:15.623759 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:15.623775 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:15.624471 (+ 696us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:15.624526 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:15.624549 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:15.624638 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:15.624897 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 07:07:15.625022 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:07:15.626646 12146 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:15.628279 12669 ts_tablet_manager.cc:1918] T 6a1a2ddb2ab14ef39c498a061ee449d5 P 44cfa5a18eac46c6865534cd7163d872: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:07:15.628477 12669 log.cc:1196] T 6a1a2ddb2ab14ef39c498a061ee449d5 P 44cfa5a18eac46c6865534cd7163d872: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/6a1a2ddb2ab14ef39c498a061ee449d5
I20260812 07:07:15.629156 12669 ts_tablet_manager.cc:1939] T 6a1a2ddb2ab14ef39c498a061ee449d5 P 44cfa5a18eac46c6865534cd7163d872: Deleting consensus metadata
I20260812 07:07:15.630864 12670 ts_tablet_manager.cc:1918] T 6a1a2ddb2ab14ef39c498a061ee449d5 P f6a900be3651454a8a12be45daf1abf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:07:15.631049 12670 log.cc:1196] T 6a1a2ddb2ab14ef39c498a061ee449d5 P f6a900be3651454a8a12be45daf1abf8: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/6a1a2ddb2ab14ef39c498a061ee449d5
I20260812 07:07:15.630971 11821 catalog_manager.cc:4841] TS 44cfa5a18eac46c6865534cd7163d872 (127.11.192.195:37793): tablet 6a1a2ddb2ab14ef39c498a061ee449d5 (table test-table [id=4a7ba75eb163478d8398a12fb6ad27bf]) successfully deleted
I20260812 07:07:15.631623 12670 ts_tablet_manager.cc:1939] T 6a1a2ddb2ab14ef39c498a061ee449d5 P f6a900be3651454a8a12be45daf1abf8: Deleting consensus metadata
I20260812 07:07:15.633169 11821 catalog_manager.cc:4841] TS f6a900be3651454a8a12be45daf1abf8 (127.11.192.194:34961): tablet 6a1a2ddb2ab14ef39c498a061ee449d5 (table test-table [id=4a7ba75eb163478d8398a12fb6ad27bf]) successfully deleted
I20260812 07:07:15.721179 12671 ts_tablet_manager.cc:1918] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:07:15.721447 12671 log.cc:1196] T 6a1a2ddb2ab14ef39c498a061ee449d5 P a1d45e34311547e79e95f278a049ae19: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518136653111-12035-0/minicluster-data/ts-0/wal/wals/6a1a2ddb2ab14ef39c498a061ee449d5
I20260812 07:07:15.721908 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 11943
I20260812 07:07:15.744203 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 12169
W20260812 07:07:15.744155 11817 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.192.193:36367 (error 108)
I20260812 07:07:15.746173 12612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.745682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:36367 (local address 127.0.0.1:51562)
0812 07:07:15.745848 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:15.745987 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:36367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 07:07:15.746414 11817 catalog_manager.cc:4584] TS a1d45e34311547e79e95f278a049ae19 (127.11.192.193:36367): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6a1a2ddb2ab14ef39c498a061ee449d5: Network error: Client connection negotiation failed: client connection to 127.11.192.193:36367: connect: Connection refused (error 111)
I20260812 07:07:15.751492 12613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.745297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49460 (local address 127.11.192.194:34961)
0812 07:07:15.745497 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:15.745503 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:15.745522 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:07:15.745587 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:15.745592 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:15.745719 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:15.745831 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:15.746338 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.747113 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.748815 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.749228 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.749325 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:15.750310 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:15.750320 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:15.750334 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:15.750870 (+ 536us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:15.750915 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:15.750936 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:15.751036 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:15.751152 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 07:07:15.751241 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 07:07:15.752126 12369 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:15.800010 12612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.799556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:36367 (local address 127.0.0.1:51566)
0812 07:07:15.799755 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:15.799845 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:36367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 07:07:15.801092 12612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.800747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:36367 (local address 127.0.0.1:51582)
0812 07:07:15.800881 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:15.800953 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:36367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 07:07:15.839022 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 12169
I20260812 07:07:15.861459 12612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.860912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:36367 (local address 127.0.0.1:51584)
0812 07:07:15.861148 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:15.861253 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:36367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 07:07:15.862490 12612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.862124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:36367 (local address 127.0.0.1:51600)
0812 07:07:15.862273 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:15.862366 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:36367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 07:07:15.864456 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 12378
I20260812 07:07:15.870894 12611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.865133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41016 (local address 127.11.192.195:37793)
0812 07:07:15.865312 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:15.865318 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:15.865339 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:07:15.865469 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:15.865473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:15.865568 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:15.865653 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:15.866102 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.866822 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.868101 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:15.868504 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:15.868616 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:15.869599 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:15.869613 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:15.869632 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:15.870203 (+ 571us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:15.870248 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:15.870268 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:15.870375 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:15.870499 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 07:07:15.870603 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:07:15.871534 12578 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:15.943274 12612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.942835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:36367 (local address 127.0.0.1:51614)
0812 07:07:15.943032 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:15.943140 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:36367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 07:07:15.944254 12612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:15.943854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:36367 (local address 127.0.0.1:51628)
0812 07:07:15.944042 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:15.944151 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:36367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260812 07:07:15.956988 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 12378
I20260812 07:07:15.986615 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 11799
I20260812 07:07:15.987764 11896 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:16.092978 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 11799
2026-08-12T07:07:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2665 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:07:16 dist-test-slave-lgr5 krb5kdc[12690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:07:16 dist-test-slave-lgr5 krb5kdc[12690](info): set up 2 sockets
Aug 12 07:07:16 dist-test-slave-lgr5 krb5kdc[12690](info): commencing operation
krb5kdc: starting...
W20260812 07:07:17.383014 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.004s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:07:17 dist-test-slave-lgr5 krb5kdc[12690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:07:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:07:17.636418 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:34569
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:34569
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:17.954957 12712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:17.955217 12712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:17.955341 12712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:17.962270 12712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:17.962373 12712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:17.962447 12712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:17.962529 12712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:17.962585 12712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:17.972478 12712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:34569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:34569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:17.974112 12712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:17.975766 12712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:17.987434 12717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:17.987402 12721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:17.987352 12718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:17.988159 12712 server_base.cc:1034] running on GCE node
I20260812 07:07:17.988864 12712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:17.990154 12712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:17.991353 12712 hybrid_clock.cc:648] HybridClock initialized: now 1786518437991297 us; error 45 us; skew 500 ppm
I20260812 07:07:17.991680 12712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:17 dist-test-slave-lgr5 krb5kdc[12690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518437, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:18.000447 12712 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:07:18.001940 12712 webserver.cc:460] Webserver started at http://127.11.192.254:36217/ using document root <none> and password file <none>
I20260812 07:07:18.002616 12712 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:18.002738 12712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:18.003041 12712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:18.006039 12712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "9253dd68351e462184dbd0744e0a6f2d"
format_stamp: "Formatted at 2026-08-12 07:07:17 on dist-test-slave-lgr5"
server_key: "0e3df25d91c0fcb8713e57bb3a6f0397"
server_key_iv: "6690a0eda9674105493d6844b1e3c088"
server_key_version: "encryptionkey@0"
I20260812 07:07:18.006786 12712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "9253dd68351e462184dbd0744e0a6f2d"
format_stamp: "Formatted at 2026-08-12 07:07:17 on dist-test-slave-lgr5"
server_key: "0e3df25d91c0fcb8713e57bb3a6f0397"
server_key_iv: "6690a0eda9674105493d6844b1e3c088"
server_key_version: "encryptionkey@0"
I20260812 07:07:18.011718 12712 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260812 07:07:18.014994 12733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:18.016295 12712 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:07:18.016453 12712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "9253dd68351e462184dbd0744e0a6f2d"
format_stamp: "Formatted at 2026-08-12 07:07:17 on dist-test-slave-lgr5"
server_key: "0e3df25d91c0fcb8713e57bb3a6f0397"
server_key_iv: "6690a0eda9674105493d6844b1e3c088"
server_key_version: "encryptionkey@0"
I20260812 07:07:18.016628 12712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:18.029688 12712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:18.033247 12712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:18.033497 12712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:18.050209 12712 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:34569
I20260812 07:07:18.050230 12819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:34569 every 8 connection(s)
I20260812 07:07:18.051548 12712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:18.055089 12821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:18.056351 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 12712
I20260812 07:07:18.056553 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:07:18.057140 12035 external_mini_cluster.cc:1428] Setting key 2417d877bbead6925b147d91104529bd
I20260812 07:07:18.068799 12821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d: Bootstrap starting.
Aug 12 07:07:18 dist-test-slave-lgr5 krb5kdc[12690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:07:18.072324 12821 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:18.073647 12821 log.cc:824] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:18.076743 12821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d: No bootstrap required, opened a new log
I20260812 07:07:18.080297 12824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:18.058967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39138 (local address 127.11.192.254:34569)
0812 07:07:18.059698 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:18.059708 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:18.059748 (+ 40us) server_negotiation.cc:407] Connection header received
0812 07:07:18.061379 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:18.061400 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:18.061760 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:18.062073 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:18.063831 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:18.065134 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:18.066942 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:18.067566 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:18.067684 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:18.072359 (+ 4675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:18.072369 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:18.072387 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:18.072437 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:18.076033 (+ 3596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:18.076704 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:18.076712 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:18.076721 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:18.076922 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:18.077287 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:18.077293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:18.077296 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:18.077988 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:18.078277 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:18.078573 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 07:07:18.078993 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":327,"threads_started":1}
I20260812 07:07:18.085007 12821 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9253dd68351e462184dbd0744e0a6f2d" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 34569 } }
I20260812 07:07:18.085312 12821 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:18.085384 12821 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9253dd68351e462184dbd0744e0a6f2d, State: Initialized, Role: FOLLOWER
I20260812 07:07:18.085843 12821 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9253dd68351e462184dbd0744e0a6f2d" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 34569 } }
I20260812 07:07:18.086036 12821 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:18.086150 12821 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:18.086298 12821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:18.088531 12821 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9253dd68351e462184dbd0744e0a6f2d" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 34569 } }
I20260812 07:07:18.088932 12821 leader_election.cc:302] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9253dd68351e462184dbd0744e0a6f2d; no voters:
I20260812 07:07:18.089389 12821 leader_election.cc:288] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:18.089501 12830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:18.090152 12830 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d [term 1 LEADER]: Becoming Leader. State: Replica: 9253dd68351e462184dbd0744e0a6f2d, State: Running, Role: LEADER
I20260812 07:07:18.090633 12830 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9253dd68351e462184dbd0744e0a6f2d" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 34569 } }
I20260812 07:07:18.091159 12821 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:18.093402 12832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9253dd68351e462184dbd0744e0a6f2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9253dd68351e462184dbd0744e0a6f2d" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 34569 } } }
I20260812 07:07:18.093793 12832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:18.094668 12833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9253dd68351e462184dbd0744e0a6f2d. Latest consensus state: current_term: 1 leader_uuid: "9253dd68351e462184dbd0744e0a6f2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9253dd68351e462184dbd0744e0a6f2d" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 34569 } } }
I20260812 07:07:18.094964 12833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:18.095541 12840 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:18.100445 12840 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:18.108867 12840 catalog_manager.cc:1347] Generated new cluster ID: 71fbb1daf6e0440286609382b59f4970
I20260812 07:07:18.108968 12840 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:18.132858 12840 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:18.134061 12840 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:18.141199 12840 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d: Generated new TSK 0
I20260812 07:07:18.141883 12840 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
I20260812 07:07:18.222957 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:34569
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:18.568912 12856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:18.569162 12856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:18.569268 12856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:18.576375 12856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:18.576469 12856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:18.576578 12856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:07:18.586012 12856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:34569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:18.587059 12856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:18.588266 12856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:18.600696 12865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:18.600659 12866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:18.600623 12868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:18.601501 12856 server_base.cc:1034] running on GCE node
I20260812 07:07:18.602067 12856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:18.602931 12856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:18.604158 12856 hybrid_clock.cc:648] HybridClock initialized: now 1786518438604103 us; error 44 us; skew 500 ppm
I20260812 07:07:18.604449 12856 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:18 dist-test-slave-lgr5 krb5kdc[12690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518438, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:18.607275 12856 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:07:18.608580 12856 webserver.cc:460] Webserver started at http://127.11.192.193:38543/ using document root <none> and password file <none>
I20260812 07:07:18.609174 12856 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:18.609270 12856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:18.609519 12856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:18.612273 12856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "19398249fa5b471492acc04330792728"
format_stamp: "Formatted at 2026-08-12 07:07:18 on dist-test-slave-lgr5"
server_key: "7d7a7a02f1f0ea0b2246ecbcb42e96ae"
server_key_iv: "4c3973a3aa760a8993e25c80d5b74919"
server_key_version: "encryptionkey@0"
I20260812 07:07:18.612962 12856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "19398249fa5b471492acc04330792728"
format_stamp: "Formatted at 2026-08-12 07:07:18 on dist-test-slave-lgr5"
server_key: "7d7a7a02f1f0ea0b2246ecbcb42e96ae"
server_key_iv: "4c3973a3aa760a8993e25c80d5b74919"
server_key_version: "encryptionkey@0"
I20260812 07:07:18.617615 12856 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:07:18.620808 12877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:18.622023 12856 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:07:18.622146 12856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "19398249fa5b471492acc04330792728"
format_stamp: "Formatted at 2026-08-12 07:07:18 on dist-test-slave-lgr5"
server_key: "7d7a7a02f1f0ea0b2246ecbcb42e96ae"
server_key_iv: "4c3973a3aa760a8993e25c80d5b74919"
server_key_version: "encryptionkey@0"
I20260812 07:07:18.622298 12856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:18.643805 12856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:18.647298 12856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:18.647526 12856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:18.648350 12856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:18.649516 12856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:18.649591 12856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:18.649679 12856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:18.649721 12856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:18.677331 12856 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:43639
I20260812 07:07:18.677417 13057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:43639 every 8 connection(s)
I20260812 07:07:18.678879 12856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:07:18.682827 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 12856
I20260812 07:07:18.683029 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:07:18.683463 12035 external_mini_cluster.cc:1428] Setting key 57505028dbdac021086cc6969e04bc84
Aug 12 07:07:18 dist-test-slave-lgr5 krb5kdc[12690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518438, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:07:18.698727 13064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:18.681488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:37759 (local address 127.11.192.254:34569)
0812 07:07:18.681968 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:18.681985 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:18.682844 (+ 859us) server_negotiation.cc:407] Connection header received
0812 07:07:18.684912 (+ 2068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:18.684916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:18.684978 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:18.685071 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:18.686567 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:18.687595 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:18.689091 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:18.689631 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:18.689752 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:18.693230 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:18.693235 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:18.693238 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:18.693269 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:18.695729 (+ 2460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:18.696528 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:18.696532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:18.696534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:18.696652 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:18.697057 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:18.697059 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:18.697061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:18.697306 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:18.697448 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:18.698144 (+ 696us) server_negotiation.cc:299] Negotiation successful
0812 07:07:18.698365 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":216,"threads_started":1}
I20260812 07:07:18.699548 13067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:18.682037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:34569 (local address 127.11.192.193:37759)
0812 07:07:18.682701 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:18.682739 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:18.684659 (+ 1920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:18.685334 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:18.685344 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:18.685867 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:18.686387 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:18.686403 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:18.687714 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:18.687717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:18.688951 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:18.688964 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:18.689808 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:18.689813 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:18.689992 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:18.690599 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:18.690622 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:18.693035 (+ 2413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:18.695853 (+ 2818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:18.695861 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:18.695873 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:18.696419 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:18.696742 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:18.696747 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:18.696750 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:18.696967 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:18.697464 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:18.697472 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:18.697757 (+ 285us) client_negotiation.cc:769] Sending connection context
0812 07:07:18.697975 (+ 218us) client_negotiation.cc:240] Negotiation successful
0812 07:07:18.698432 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":250,"threads_started":1}
I20260812 07:07:18.701014 13058 heartbeater.cc:347] Connected to a master server at 127.11.192.254:34569
I20260812 07:07:18.701274 13058 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:18.701910 13058 heartbeater.cc:511] Master 127.11.192.254:34569 requested a full tablet report, sending...
I20260812 07:07:18.704160 12759 ts_manager.cc:194] Registered new tserver with Master: 19398249fa5b471492acc04330792728 (127.11.192.193:43639)
I20260812 07:07:18.706028 12759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:37759
WARNING: no policy specified for kudu/127.11.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
I20260812 07:07:18.746243 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:34569
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:19.069778 13072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:19.070016 13072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:19.070106 13072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:19.077059 13072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:19.077157 13072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:19.077273 13072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:07:19.086503 13072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:34569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:19.087564 13072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:19.088848 13072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:19.101107 13090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:19.101150 13081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:19.101176 13082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:19.101962 13072 server_base.cc:1034] running on GCE node
I20260812 07:07:19.102499 13072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:19.103319 13072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:19.104516 13072 hybrid_clock.cc:648] HybridClock initialized: now 1786518439104470 us; error 39 us; skew 500 ppm
I20260812 07:07:19.104799 13072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:19 dist-test-slave-lgr5 krb5kdc[12690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518439, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:19.107635 13072 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:07:19.109000 13072 webserver.cc:460] Webserver started at http://127.11.192.194:43909/ using document root <none> and password file <none>
I20260812 07:07:19.109616 13072 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:19.109733 13072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:19.110067 13072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:19.112810 13072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "b3dc54b2371e4a9ea40d19483db27c48"
format_stamp: "Formatted at 2026-08-12 07:07:19 on dist-test-slave-lgr5"
server_key: "5a134de5b5ffb7f908596ca97d6c2f75"
server_key_iv: "70fe8e6aa80e0387cd1025109b5cc573"
server_key_version: "encryptionkey@0"
I20260812 07:07:19.113530 13072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "b3dc54b2371e4a9ea40d19483db27c48"
format_stamp: "Formatted at 2026-08-12 07:07:19 on dist-test-slave-lgr5"
server_key: "5a134de5b5ffb7f908596ca97d6c2f75"
server_key_iv: "70fe8e6aa80e0387cd1025109b5cc573"
server_key_version: "encryptionkey@0"
I20260812 07:07:19.118312 13072 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 07:07:19.121532 13099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:19.122699 13072 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:07:19.122848 13072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "b3dc54b2371e4a9ea40d19483db27c48"
format_stamp: "Formatted at 2026-08-12 07:07:19 on dist-test-slave-lgr5"
server_key: "5a134de5b5ffb7f908596ca97d6c2f75"
server_key_iv: "70fe8e6aa80e0387cd1025109b5cc573"
server_key_version: "encryptionkey@0"
I20260812 07:07:19.123016 13072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:19.143719 13072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:19.147289 13072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:19.147540 13072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:19.148425 13072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:19.149700 13072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:19.149782 13072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:19.149895 13072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:19.149947 13072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:19.180938 13072 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:38745
I20260812 07:07:19.180968 13301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:38745 every 8 connection(s)
I20260812 07:07:19.182214 13072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:07:19.189777 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 13072
I20260812 07:07:19.189940 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:07:19.190371 12035 external_mini_cluster.cc:1428] Setting key 703967cf9fd59dd3227346835746055f
Aug 12 07:07:19 dist-test-slave-lgr5 krb5kdc[12690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518439, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:07:19.202540 13064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:19.184722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:40011 (local address 127.11.192.254:34569)
0812 07:07:19.184932 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:19.184940 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:19.186122 (+ 1182us) server_negotiation.cc:407] Connection header received
0812 07:07:19.187837 (+ 1715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:19.187842 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:19.187916 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:19.188007 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:19.189387 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.190197 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.192043 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.192740 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.192869 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:19.196534 (+ 3665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:19.196541 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:19.196545 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:19.196582 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:19.199531 (+ 2949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:19.200381 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:19.200386 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:19.200389 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:19.200561 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:19.200967 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:19.200972 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:19.200975 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:19.201292 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:19.201426 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:19.202031 (+ 605us) server_negotiation.cc:299] Negotiation successful
0812 07:07:19.202235 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:07:19.203213 13305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:19.185249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:34569 (local address 127.11.192.194:40011)
0812 07:07:19.185950 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:19.186007 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:19.187587 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:19.188235 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:19.188244 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:19.188712 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:19.189248 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:19.189265 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.190374 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.190379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:19.191897 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:19.191913 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.192958 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.192964 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:19.193165 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:19.193828 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:19.193859 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:19.196310 (+ 2451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:19.199678 (+ 3368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:19.199685 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:19.199696 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:19.200266 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:19.200662 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:19.200665 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:19.200668 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:19.200858 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:19.201401 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:19.201406 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:19.201673 (+ 267us) client_negotiation.cc:769] Sending connection context
0812 07:07:19.201886 (+ 213us) client_negotiation.cc:240] Negotiation successful
0812 07:07:19.202223 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":272,"threads_started":1}
I20260812 07:07:19.204522 13302 heartbeater.cc:347] Connected to a master server at 127.11.192.254:34569
I20260812 07:07:19.204783 13302 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:19.205475 13302 heartbeater.cc:511] Master 127.11.192.254:34569 requested a full tablet report, sending...
I20260812 07:07:19.206854 12759 ts_manager.cc:194] Registered new tserver with Master: b3dc54b2371e4a9ea40d19483db27c48 (127.11.192.194:38745)
I20260812 07:07:19.207629 12759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:40011
WARNING: no policy specified for kudu/127.11.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
I20260812 07:07:19.253003 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:34569
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:19.577419 13310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:19.577708 13310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:19.577867 13310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:19.585098 13310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:19.585227 13310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:19.585404 13310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:07:19.595060 13310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:34569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:19.596379 13310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:19.597673 13310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:19.610123 13322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:19.610157 13323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:19.610169 13326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:19.611135 13310 server_base.cc:1034] running on GCE node
I20260812 07:07:19.611730 13310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:19.612686 13310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:19.613893 13310 hybrid_clock.cc:648] HybridClock initialized: now 1786518439613843 us; error 39 us; skew 500 ppm
I20260812 07:07:19.614197 13310 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:19 dist-test-slave-lgr5 krb5kdc[12690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518439, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:19.617347 13310 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:07:19.618777 13310 webserver.cc:460] Webserver started at http://127.11.192.195:42389/ using document root <none> and password file <none>
I20260812 07:07:19.619432 13310 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:19.619560 13310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:19.619870 13310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:19.622834 13310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "5d933cc61f634f0aa4d15905984bb61a"
format_stamp: "Formatted at 2026-08-12 07:07:19 on dist-test-slave-lgr5"
server_key: "f712cc55bed0ff0a1d46684f634b0014"
server_key_iv: "4b60033ea9dde21d610c45e7d79a3b9c"
server_key_version: "encryptionkey@0"
I20260812 07:07:19.623616 13310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "5d933cc61f634f0aa4d15905984bb61a"
format_stamp: "Formatted at 2026-08-12 07:07:19 on dist-test-slave-lgr5"
server_key: "f712cc55bed0ff0a1d46684f634b0014"
server_key_iv: "4b60033ea9dde21d610c45e7d79a3b9c"
server_key_version: "encryptionkey@0"
I20260812 07:07:19.629048 13310 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 07:07:19.632457 13335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:19.633666 13310 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 07:07:19.633821 13310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "5d933cc61f634f0aa4d15905984bb61a"
format_stamp: "Formatted at 2026-08-12 07:07:19 on dist-test-slave-lgr5"
server_key: "f712cc55bed0ff0a1d46684f634b0014"
server_key_iv: "4b60033ea9dde21d610c45e7d79a3b9c"
server_key_version: "encryptionkey@0"
I20260812 07:07:19.633991 13310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:19.648839 13310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:19.652603 13310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:19.652850 13310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:19.653689 13310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:19.654876 13310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:19.654953 13310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:19.655077 13310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:19.655135 13310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:19.686137 13310 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:41427
I20260812 07:07:19.686179 13527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:41427 every 8 connection(s)
I20260812 07:07:19.687294 13310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:07:19.695022 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 13310
I20260812 07:07:19.695211 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:07:19.695621 12035 external_mini_cluster.cc:1428] Setting key dd38e67f94fad520376c426549612a3e
Aug 12 07:07:19 dist-test-slave-lgr5 krb5kdc[12690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518439, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:07:19.706220 13064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:19.689679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:45673 (local address 127.11.192.254:34569)
0812 07:07:19.689845 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:19.689849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:19.690896 (+ 1047us) server_negotiation.cc:407] Connection header received
0812 07:07:19.692987 (+ 2091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:19.692992 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:19.693090 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:19.693176 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:19.694599 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.695496 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.697447 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.697856 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.697969 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:19.701175 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:19.701180 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:19.701183 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:19.701215 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:19.703416 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:19.704069 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:19.704073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:19.704075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:19.704238 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:19.704648 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:19.704652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:19.704654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:19.704915 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:19.705056 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:19.705671 (+ 615us) server_negotiation.cc:299] Negotiation successful
0812 07:07:19.705885 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:07:19.707002 13531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:19.690055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:34569 (local address 127.11.192.195:45673)
0812 07:07:19.690738 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:19.690772 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:19.692713 (+ 1941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:19.693392 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:19.693399 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:19.693898 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:19.694452 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:19.694469 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.695653 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.695659 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:19.697306 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:19.697319 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.698068 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.698077 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:19.698318 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:19.698900 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:19.698920 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:19.700985 (+ 2065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:19.703557 (+ 2572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:19.703565 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:19.703578 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:19.703944 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:19.704343 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:19.704348 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:19.704352 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:19.704533 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:19.705036 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:19.705044 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:19.705323 (+ 279us) client_negotiation.cc:769] Sending connection context
0812 07:07:19.705547 (+ 224us) client_negotiation.cc:240] Negotiation successful
0812 07:07:19.705999 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":258,"threads_started":1}
I20260812 07:07:19.708280 13529 heartbeater.cc:347] Connected to a master server at 127.11.192.254:34569
I20260812 07:07:19.708552 13529 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:19.708835 13058 heartbeater.cc:503] Master 127.11.192.254:34569 was elected leader, sending a full tablet report...
I20260812 07:07:19.709362 13529 heartbeater.cc:511] Master 127.11.192.254:34569 requested a full tablet report, sending...
I20260812 07:07:19.710781 12759 ts_manager.cc:194] Registered new tserver with Master: 5d933cc61f634f0aa4d15905984bb61a (127.11.192.195:41427)
I20260812 07:07:19.711488 12759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:45673
I20260812 07:07:19.722193 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:07:19.738318 13064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:19.726080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48730 (local address 127.11.192.254:34569)
0812 07:07:19.726229 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:19.726233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:19.726734 (+ 501us) server_negotiation.cc:407] Connection header received
0812 07:07:19.726881 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:19.726887 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:19.726940 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:19.727025 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:19.728577 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.729239 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.730492 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.730865 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.730969 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:19.733438 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:19.733443 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:19.733446 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:19.733477 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:19.735620 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:19.736364 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:19.736368 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:19.736370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:19.736495 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:19.736940 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:19.736944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:19.736946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:19.737253 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:19.737500 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:19.737933 (+ 433us) server_negotiation.cc:299] Negotiation successful
0812 07:07:19.738102 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:07:19.741539 12759 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48730:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:07:19.748008 12759 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:07:19.784397 13559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:19.759905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52708 (local address 127.11.192.194:38745)
0812 07:07:19.775419 (+ 15514us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:19.775427 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:19.777009 (+ 1582us) server_negotiation.cc:407] Connection header received
0812 07:07:19.777097 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:19.777102 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:19.777323 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:19.777507 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:19.778478 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.779618 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.782226 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.783325 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.783482 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:19.783665 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:19.783906 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 07:07:19.784073 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15343,"thread_start_us":15206,"threads_started":1}
I20260812 07:07:19.784483 13548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:19.758704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:38745 (local address 127.0.0.1:52708)
0812 07:07:19.776754 (+ 18050us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:19.776771 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:19.776883 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:19.777792 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:19.777800 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:19.777841 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:19.778274 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:19.778287 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.779743 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.779748 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:19.782055 (+ 2307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:19.782074 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.783503 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.783509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:19.783687 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:19.783733 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 07:07:19.783847 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 07:07:19.784028 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17917,"spinlock_wait_cycles":4480,"thread_start_us":17754,"threads_started":1}
I20260812 07:07:19.785140 13547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:19.759510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:41427 (local address 127.0.0.1:52254)
0812 07:07:19.777085 (+ 17575us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:19.777104 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:19.777237 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:19.777792 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:19.777800 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:19.777854 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:19.778392 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:19.778407 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.780323 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.780329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:19.782373 (+ 2044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:19.782384 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.784337 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.784345 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:19.784511 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:19.784541 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 07:07:19.784629 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 07:07:19.784751 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17432,"spinlock_wait_cycles":5376,"thread_start_us":16059,"threads_started":1}
I20260812 07:07:19.785355 13554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:19.758448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:43639 (local address 127.0.0.1:43610)
0812 07:07:19.775928 (+ 17480us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:19.775981 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:19.776193 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:19.778344 (+ 2151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:19.778348 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:19.778376 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:19.778774 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:19.778786 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.780550 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.780554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:19.782599 (+ 2045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:19.782614 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.784689 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.784695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:19.784856 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:19.784887 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 07:07:19.784967 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 07:07:19.785079 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17251,"thread_start_us":18342,"threads_started":1}
I20260812 07:07:19.785430 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:19.759716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52254 (local address 127.11.192.195:41427)
0812 07:07:19.772719 (+ 13003us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:19.772728 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:19.777213 (+ 4485us) server_negotiation.cc:407] Connection header received
0812 07:07:19.777366 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:19.777373 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:19.777595 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:19.777775 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:19.778639 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.780170 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.782512 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.783806 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.783984 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:19.784178 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:19.784981 (+ 803us) server_negotiation.cc:299] Negotiation successful
0812 07:07:19.785138 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12813,"thread_start_us":13017,"threads_started":1}
I20260812 07:07:19.786645 13552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:19.758739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43610 (local address 127.11.192.193:43639)
0812 07:07:19.777791 (+ 19052us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:19.777799 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:19.777823 (+ 24us) server_negotiation.cc:407] Connection header received
0812 07:07:19.777923 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:19.777930 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:19.778147 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:19.778341 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:19.778952 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.780409 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.782778 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.784205 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.784385 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:19.784545 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:19.785555 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0812 07:07:19.786407 (+ 852us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18901,"thread_start_us":18760,"threads_started":1}
I20260812 07:07:19.788878 13409 tablet_service.cc:1467] Processing CreateTablet for tablet b4937e600fe940648d883312506eb398 (DEFAULT_TABLE table=test-table [id=56b52d1b2bf64ebbb6a54e8f819f1936]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:19.788878 13182 tablet_service.cc:1467] Processing CreateTablet for tablet b4937e600fe940648d883312506eb398 (DEFAULT_TABLE table=test-table [id=56b52d1b2bf64ebbb6a54e8f819f1936]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:19.789567 12952 tablet_service.cc:1467] Processing CreateTablet for tablet b4937e600fe940648d883312506eb398 (DEFAULT_TABLE table=test-table [id=56b52d1b2bf64ebbb6a54e8f819f1936]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:19.790339 13182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4937e600fe940648d883312506eb398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:19.790340 13409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4937e600fe940648d883312506eb398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:19.790688 12952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4937e600fe940648d883312506eb398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:19.801481 13561 tablet_bootstrap.cc:492] T b4937e600fe940648d883312506eb398 P 19398249fa5b471492acc04330792728: Bootstrap starting.
I20260812 07:07:19.804316 13562 tablet_bootstrap.cc:492] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48: Bootstrap starting.
I20260812 07:07:19.805132 13563 tablet_bootstrap.cc:492] T b4937e600fe940648d883312506eb398 P 5d933cc61f634f0aa4d15905984bb61a: Bootstrap starting.
I20260812 07:07:19.805485 13561 tablet_bootstrap.cc:654] T b4937e600fe940648d883312506eb398 P 19398249fa5b471492acc04330792728: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:19.806512 13561 log.cc:824] T b4937e600fe940648d883312506eb398 P 19398249fa5b471492acc04330792728: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:19.808527 13563 tablet_bootstrap.cc:654] T b4937e600fe940648d883312506eb398 P 5d933cc61f634f0aa4d15905984bb61a: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:19.808570 13562 tablet_bootstrap.cc:654] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:19.809574 13561 tablet_bootstrap.cc:492] T b4937e600fe940648d883312506eb398 P 19398249fa5b471492acc04330792728: No bootstrap required, opened a new log
I20260812 07:07:19.809705 13562 log.cc:824] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:19.809705 13563 log.cc:824] T b4937e600fe940648d883312506eb398 P 5d933cc61f634f0aa4d15905984bb61a: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:19.809815 13561 ts_tablet_manager.cc:1397] T b4937e600fe940648d883312506eb398 P 19398249fa5b471492acc04330792728: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 07:07:19.812244 13562 tablet_bootstrap.cc:492] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48: No bootstrap required, opened a new log
I20260812 07:07:19.812474 13562 ts_tablet_manager.cc:1397] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260812 07:07:19.812707 13563 tablet_bootstrap.cc:492] T b4937e600fe940648d883312506eb398 P 5d933cc61f634f0aa4d15905984bb61a: No bootstrap required, opened a new log
I20260812 07:07:19.812942 13563 ts_tablet_manager.cc:1397] T b4937e600fe940648d883312506eb398 P 5d933cc61f634f0aa4d15905984bb61a: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260812 07:07:19.819013 13562 raft_consensus.cc:348] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19398249fa5b471492acc04330792728" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43639 } } peers { permanent_uuid: "b3dc54b2371e4a9ea40d19483db27c48" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38745 } } peers { permanent_uuid: "5d933cc61f634f0aa4d15905984bb61a" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41427 } }
I20260812 07:07:19.819332 13562 raft_consensus.cc:374] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:19.819439 13562 raft_consensus.cc:729] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3dc54b2371e4a9ea40d19483db27c48, State: Initialized, Role: FOLLOWER
I20260812 07:07:19.820019 13562 consensus_queue.cc:260] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19398249fa5b471492acc04330792728" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43639 } } peers { permanent_uuid: "b3dc54b2371e4a9ea40d19483db27c48" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38745 } } peers { permanent_uuid: "5d933cc61f634f0aa4d15905984bb61a" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41427 } }
I20260812 07:07:19.819909 13561 raft_consensus.cc:348] T b4937e600fe940648d883312506eb398 P 19398249fa5b471492acc04330792728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19398249fa5b471492acc04330792728" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43639 } } peers { permanent_uuid: "b3dc54b2371e4a9ea40d19483db27c48" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38745 } } peers { permanent_uuid: "5d933cc61f634f0aa4d15905984bb61a" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41427 } }
I20260812 07:07:19.821307 13302 heartbeater.cc:503] Master 127.11.192.254:34569 was elected leader, sending a full tablet report...
I20260812 07:07:19.820392 13561 raft_consensus.cc:374] T b4937e600fe940648d883312506eb398 P 19398249fa5b471492acc04330792728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:19.821425 13561 raft_consensus.cc:729] T b4937e600fe940648d883312506eb398 P 19398249fa5b471492acc04330792728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19398249fa5b471492acc04330792728, State: Initialized, Role: FOLLOWER
I20260812 07:07:19.821905 13562 ts_tablet_manager.cc:1428] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48: Time spent starting tablet: real 0.009s user 0.006s sys 0.003s
I20260812 07:07:19.822077 13561 consensus_queue.cc:260] T b4937e600fe940648d883312506eb398 P 19398249fa5b471492acc04330792728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19398249fa5b471492acc04330792728" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43639 } } peers { permanent_uuid: "b3dc54b2371e4a9ea40d19483db27c48" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38745 } } peers { permanent_uuid: "5d933cc61f634f0aa4d15905984bb61a" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41427 } }
I20260812 07:07:19.822877 13563 raft_consensus.cc:348] T b4937e600fe940648d883312506eb398 P 5d933cc61f634f0aa4d15905984bb61a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19398249fa5b471492acc04330792728" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43639 } } peers { permanent_uuid: "b3dc54b2371e4a9ea40d19483db27c48" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38745 } } peers { permanent_uuid: "5d933cc61f634f0aa4d15905984bb61a" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41427 } }
I20260812 07:07:19.823354 13563 raft_consensus.cc:374] T b4937e600fe940648d883312506eb398 P 5d933cc61f634f0aa4d15905984bb61a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:19.823470 13563 raft_consensus.cc:729] T b4937e600fe940648d883312506eb398 P 5d933cc61f634f0aa4d15905984bb61a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d933cc61f634f0aa4d15905984bb61a, State: Initialized, Role: FOLLOWER
I20260812 07:07:19.823827 13561 ts_tablet_manager.cc:1428] T b4937e600fe940648d883312506eb398 P 19398249fa5b471492acc04330792728: Time spent starting tablet: real 0.014s user 0.004s sys 0.008s
I20260812 07:07:19.824149 13563 consensus_queue.cc:260] T b4937e600fe940648d883312506eb398 P 5d933cc61f634f0aa4d15905984bb61a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19398249fa5b471492acc04330792728" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43639 } } peers { permanent_uuid: "b3dc54b2371e4a9ea40d19483db27c48" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38745 } } peers { permanent_uuid: "5d933cc61f634f0aa4d15905984bb61a" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41427 } }
I20260812 07:07:19.825328 13529 heartbeater.cc:503] Master 127.11.192.254:34569 was elected leader, sending a full tablet report...
I20260812 07:07:19.825855 13563 ts_tablet_manager.cc:1428] T b4937e600fe940648d883312506eb398 P 5d933cc61f634f0aa4d15905984bb61a: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260812 07:07:19.886795 13570 raft_consensus.cc:482] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:07:19.887074 13570 raft_consensus.cc:504] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19398249fa5b471492acc04330792728" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43639 } } peers { permanent_uuid: "b3dc54b2371e4a9ea40d19483db27c48" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38745 } } peers { permanent_uuid: "5d933cc61f634f0aa4d15905984bb61a" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41427 } }
I20260812 07:07:19.888700 13570 leader_election.cc:288] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19398249fa5b471492acc04330792728 (127.11.192.193:43639), 5d933cc61f634f0aa4d15905984bb61a (127.11.192.195:41427)
I20260812 07:07:19.895285 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:19.889165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:48669 (local address 127.11.192.195:41427)
0812 07:07:19.889353 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:19.889359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:19.890068 (+ 709us) server_negotiation.cc:407] Connection header received
0812 07:07:19.890245 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:19.890250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:19.890309 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:19.890385 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:19.890990 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.891712 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.893627 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.894372 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.894519 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:19.894593 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:19.894900 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 07:07:19.895035 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:07:19.895413 13576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:19.889030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:41427 (local address 127.11.192.194:48669)
0812 07:07:19.889958 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:19.889973 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:19.890113 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:19.890416 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:19.890421 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:19.890445 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:19.890840 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:19.890850 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.891829 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.891835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:19.893465 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:19.893487 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.894577 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.894584 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:19.894723 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:19.894770 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 07:07:19.894862 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 07:07:19.894993 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"mutex_wait_us":58,"thread_start_us":605,"threads_started":1}
I20260812 07:07:19.895411 13552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:19.889108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:33443 (local address 127.11.192.193:43639)
0812 07:07:19.889308 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:19.889313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:19.890145 (+ 832us) server_negotiation.cc:407] Connection header received
0812 07:07:19.890296 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:19.890300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:19.890356 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:19.890426 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:19.891075 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.891846 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.893652 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.894536 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.894683 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:19.894753 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:19.895055 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 07:07:19.895175 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:07:19.895993 13573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:19.889036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:43639 (local address 127.11.192.194:33443)
0812 07:07:19.890039 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:19.890053 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:19.890171 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:19.890454 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:19.890459 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:19.890482 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:19.890932 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:19.890943 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.891990 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.891996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:19.893503 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:19.893516 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.894728 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.894734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:19.894890 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:19.894914 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 07:07:19.894991 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 07:07:19.895176 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":909,"mutex_wait_us":71,"thread_start_us":518,"threads_started":1}
I20260812 07:07:19.896380 13442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4937e600fe940648d883312506eb398" candidate_uuid: "b3dc54b2371e4a9ea40d19483db27c48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d933cc61f634f0aa4d15905984bb61a" is_pre_election: true
I20260812 07:07:19.896896 13442 raft_consensus.cc:2454] T b4937e600fe940648d883312506eb398 P 5d933cc61f634f0aa4d15905984bb61a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3dc54b2371e4a9ea40d19483db27c48 in term 0.
I20260812 07:07:19.897015 12985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4937e600fe940648d883312506eb398" candidate_uuid: "b3dc54b2371e4a9ea40d19483db27c48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19398249fa5b471492acc04330792728" is_pre_election: true
I20260812 07:07:19.897531 12985 raft_consensus.cc:2454] T b4937e600fe940648d883312506eb398 P 19398249fa5b471492acc04330792728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3dc54b2371e4a9ea40d19483db27c48 in term 0.
I20260812 07:07:19.897529 13102 leader_election.cc:302] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d933cc61f634f0aa4d15905984bb61a, b3dc54b2371e4a9ea40d19483db27c48; no voters:
I20260812 07:07:19.897934 13570 raft_consensus.cc:2781] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:07:19.898097 13570 raft_consensus.cc:482] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:07:19.898208 13570 raft_consensus.cc:3037] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:19.900517 13570 raft_consensus.cc:504] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19398249fa5b471492acc04330792728" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43639 } } peers { permanent_uuid: "b3dc54b2371e4a9ea40d19483db27c48" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38745 } } peers { permanent_uuid: "5d933cc61f634f0aa4d15905984bb61a" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41427 } }
I20260812 07:07:19.901031 13570 leader_election.cc:288] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48 [CANDIDATE]: Term 1 election: Requested vote from peers 19398249fa5b471492acc04330792728 (127.11.192.193:43639), 5d933cc61f634f0aa4d15905984bb61a (127.11.192.195:41427)
I20260812 07:07:19.901422 12985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4937e600fe940648d883312506eb398" candidate_uuid: "b3dc54b2371e4a9ea40d19483db27c48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19398249fa5b471492acc04330792728"
I20260812 07:07:19.901464 13442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4937e600fe940648d883312506eb398" candidate_uuid: "b3dc54b2371e4a9ea40d19483db27c48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d933cc61f634f0aa4d15905984bb61a"
I20260812 07:07:19.901587 12985 raft_consensus.cc:3037] T b4937e600fe940648d883312506eb398 P 19398249fa5b471492acc04330792728 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:19.901639 13442 raft_consensus.cc:3037] T b4937e600fe940648d883312506eb398 P 5d933cc61f634f0aa4d15905984bb61a [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:19.903723 12985 raft_consensus.cc:2454] T b4937e600fe940648d883312506eb398 P 19398249fa5b471492acc04330792728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3dc54b2371e4a9ea40d19483db27c48 in term 1.
I20260812 07:07:19.903743 13442 raft_consensus.cc:2454] T b4937e600fe940648d883312506eb398 P 5d933cc61f634f0aa4d15905984bb61a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3dc54b2371e4a9ea40d19483db27c48 in term 1.
I20260812 07:07:19.904071 13102 leader_election.cc:302] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d933cc61f634f0aa4d15905984bb61a, b3dc54b2371e4a9ea40d19483db27c48; no voters:
I20260812 07:07:19.904384 13570 raft_consensus.cc:2781] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:19.904812 13570 raft_consensus.cc:686] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48 [term 1 LEADER]: Becoming Leader. State: Replica: b3dc54b2371e4a9ea40d19483db27c48, State: Running, Role: LEADER
I20260812 07:07:19.905437 13570 consensus_queue.cc:237] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19398249fa5b471492acc04330792728" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43639 } } peers { permanent_uuid: "b3dc54b2371e4a9ea40d19483db27c48" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38745 } } peers { permanent_uuid: "5d933cc61f634f0aa4d15905984bb61a" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41427 } }
I20260812 07:07:19.909914 12759 catalog_manager.cc:5495] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3dc54b2371e4a9ea40d19483db27c48 (127.11.192.194). New cstate: current_term: 1 leader_uuid: "b3dc54b2371e4a9ea40d19483db27c48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19398249fa5b471492acc04330792728" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3dc54b2371e4a9ea40d19483db27c48" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 38745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d933cc61f634f0aa4d15905984bb61a" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41427 } health_report { overall_health: UNKNOWN } } }
W20260812 07:07:19.931566 13061 tablet.cc:2369] T b4937e600fe940648d883312506eb398 P 19398249fa5b471492acc04330792728: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:07:19.934047 13303 tablet.cc:2369] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:07:19.938750 13530 tablet.cc:2369] T b4937e600fe940648d883312506eb398 P 5d933cc61f634f0aa4d15905984bb61a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:07:19.957197 13559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:19.952400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52714 (local address 127.11.192.194:38745)
0812 07:07:19.952631 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:19.952636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:19.952653 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:07:19.952818 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:19.952821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:19.952905 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:19.953027 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:07:19.953331 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.954034 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.955316 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:19.955690 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:19.955778 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:19.956222 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:07:19.956686 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:07:19.956812 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:19.956895 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 07:07:19.956932 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 07:07:19.964572 13442 raft_consensus.cc:1261] T b4937e600fe940648d883312506eb398 P 5d933cc61f634f0aa4d15905984bb61a [term 1 FOLLOWER]: Refusing update from remote peer b3dc54b2371e4a9ea40d19483db27c48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:07:19.964578 12985 raft_consensus.cc:1261] T b4937e600fe940648d883312506eb398 P 19398249fa5b471492acc04330792728 [term 1 FOLLOWER]: Refusing update from remote peer b3dc54b2371e4a9ea40d19483db27c48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:07:19.965444 13570 consensus_queue.cc:1035] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d933cc61f634f0aa4d15905984bb61a" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:07:19.965711 13578 consensus_queue.cc:1035] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19398249fa5b471492acc04330792728" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 43639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:07:19.974941 13582 mvcc.cc:205] Tried to move back new op lower bound from 7317579530085720064 to 7317579529856139264. Current Snapshot: MvccSnapshot[applied={T|T < 7317579530085720064}]
I20260812 07:07:19.977970 13583 mvcc.cc:205] Tried to move back new op lower bound from 7317579530085720064 to 7317579529856139264. Current Snapshot: MvccSnapshot[applied={T|T < 7317579530085720064}]
I20260812 07:07:19.985565 13584 mvcc.cc:205] Tried to move back new op lower bound from 7317579530085720064 to 7317579529856139264. Current Snapshot: MvccSnapshot[applied={T|T < 7317579530085720064 or (T in {7317579530085720064})}]
I20260812 07:07:19.991291 12759 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:07:19.991554 12759 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:07:19.995033 12759 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9253dd68351e462184dbd0744e0a6f2d: Sending DeleteTablet for 3 replicas of tablet b4937e600fe940648d883312506eb398
I20260812 07:07:19.996117 13182 tablet_service.cc:1514] Processing DeleteTablet for tablet b4937e600fe940648d883312506eb398 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:19 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:52708
I20260812 07:07:19.996107 12952 tablet_service.cc:1514] Processing DeleteTablet for tablet b4937e600fe940648d883312506eb398 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:19 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:43610
I20260812 07:07:19.996224 13409 tablet_service.cc:1514] Processing DeleteTablet for tablet b4937e600fe940648d883312506eb398 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:19 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:52254
I20260812 07:07:20.000449 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 12856
I20260812 07:07:20.008574 13599 tablet_replica.cc:330] T b4937e600fe940648d883312506eb398 P 5d933cc61f634f0aa4d15905984bb61a: stopping tablet replica
I20260812 07:07:20.009024 13599 raft_consensus.cc:2229] T b4937e600fe940648d883312506eb398 P 5d933cc61f634f0aa4d15905984bb61a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:07:20.009546 13599 raft_consensus.cc:2258] T b4937e600fe940648d883312506eb398 P 5d933cc61f634f0aa4d15905984bb61a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:20.009712 13596 tablet_replica.cc:330] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48: stopping tablet replica
I20260812 07:07:20.010773 13596 raft_consensus.cc:2229] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:07:20.011354 13596 raft_consensus.cc:2258] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:20.013662 13596 ts_tablet_manager.cc:1905] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:20.014387 13599 ts_tablet_manager.cc:1905] T b4937e600fe940648d883312506eb398 P 5d933cc61f634f0aa4d15905984bb61a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:20.015930 13597 tablet_replica.cc:330] T b4937e600fe940648d883312506eb398 P 19398249fa5b471492acc04330792728: stopping tablet replica
I20260812 07:07:20.019511 13597 raft_consensus.cc:2229] T b4937e600fe940648d883312506eb398 P 19398249fa5b471492acc04330792728 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:07:20.020123 13597 raft_consensus.cc:2258] T b4937e600fe940648d883312506eb398 P 19398249fa5b471492acc04330792728 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:20.021160 13596 ts_tablet_manager.cc:1918] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:07:20.021381 13596 log.cc:1196] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/b4937e600fe940648d883312506eb398
I20260812 07:07:20.022039 13596 ts_tablet_manager.cc:1939] T b4937e600fe940648d883312506eb398 P b3dc54b2371e4a9ea40d19483db27c48: Deleting consensus metadata
I20260812 07:07:20.022324 13597 ts_tablet_manager.cc:1905] T b4937e600fe940648d883312506eb398 P 19398249fa5b471492acc04330792728: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:20.024147 12738 catalog_manager.cc:4841] TS b3dc54b2371e4a9ea40d19483db27c48 (127.11.192.194:38745): tablet b4937e600fe940648d883312506eb398 (table test-table [id=56b52d1b2bf64ebbb6a54e8f819f1936]) successfully deleted
Aug 12 07:07:20 dist-test-slave-lgr5 krb5kdc[12690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.193@KRBTEST.COM
I20260812 07:07:20.026871 13599 ts_tablet_manager.cc:1918] T b4937e600fe940648d883312506eb398 P 5d933cc61f634f0aa4d15905984bb61a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:07:20.027016 13599 log.cc:1196] T b4937e600fe940648d883312506eb398 P 5d933cc61f634f0aa4d15905984bb61a: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/b4937e600fe940648d883312506eb398
I20260812 07:07:20.027436 13599 ts_tablet_manager.cc:1939] T b4937e600fe940648d883312506eb398 P 5d933cc61f634f0aa4d15905984bb61a: Deleting consensus metadata
I20260812 07:07:20.028957 12736 catalog_manager.cc:4841] TS 5d933cc61f634f0aa4d15905984bb61a (127.11.192.195:41427): tablet b4937e600fe940648d883312506eb398 (table test-table [id=56b52d1b2bf64ebbb6a54e8f819f1936]) successfully deleted
I20260812 07:07:20.032362 13597 ts_tablet_manager.cc:1918] T b4937e600fe940648d883312506eb398 P 19398249fa5b471492acc04330792728: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:07:20.032496 13597 log.cc:1196] T b4937e600fe940648d883312506eb398 P 19398249fa5b471492acc04330792728: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518136653111-12035-0/minicluster-data/ts-0/wal/wals/b4937e600fe940648d883312506eb398
I20260812 07:07:20.032907 13597 ts_tablet_manager.cc:1939] T b4937e600fe940648d883312506eb398 P 19398249fa5b471492acc04330792728: Deleting consensus metadata
I20260812 07:07:20.034162 12735 catalog_manager.cc:4841] TS 19398249fa5b471492acc04330792728 (127.11.192.193:43639): tablet b4937e600fe940648d883312506eb398 (table test-table [id=56b52d1b2bf64ebbb6a54e8f819f1936]) successfully deleted
I20260812 07:07:20.034685 13552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:20.001618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43618 (local address 127.11.192.193:43639)
0812 07:07:20.001800 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:20.001807 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:20.015120 (+ 13313us) server_negotiation.cc:407] Connection header received
0812 07:07:20.015587 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:20.015594 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:20.015679 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:20.015776 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:20.017557 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:20.018693 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:20.020769 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:20.021460 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:20.021568 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:20.027225 (+ 5657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:20.027240 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:20.027256 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:20.027301 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:20.031085 (+ 3784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:20.031932 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:20.031941 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:20.031944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:20.032158 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:20.032663 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:20.032668 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:20.032676 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:20.033144 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:20.033461 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:20.034060 (+ 599us) server_negotiation.cc:299] Negotiation successful
0812 07:07:20.034302 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:07:20.035933 13046 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:20.136130 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 12856
I20260812 07:07:20.163328 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 13072
Aug 12 07:07:20 dist-test-slave-lgr5 krb5kdc[12690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.194@KRBTEST.COM
I20260812 07:07:20.179788 13559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:20.164064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52718 (local address 127.11.192.194:38745)
0812 07:07:20.164264 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:20.164270 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:20.164357 (+ 87us) server_negotiation.cc:407] Connection header received
0812 07:07:20.164551 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:20.164556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:20.164639 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:20.164735 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:20.166003 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:20.166847 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:20.168222 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:20.168760 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:20.168884 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:20.174014 (+ 5130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:20.174027 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:20.174041 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:20.174085 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:20.177096 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:20.177903 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:20.177910 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:20.177913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:20.178042 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:20.178408 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:20.178411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:20.178413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:20.178756 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:20.179000 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:20.179328 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 07:07:20.179486 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260812 07:07:20.180611 13290 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:20.291882 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 13072
I20260812 07:07:20.315784 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 13310
Aug 12 07:07:20 dist-test-slave-lgr5 krb5kdc[12690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.195@KRBTEST.COM
I20260812 07:07:20.329929 13558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:20.316542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52256 (local address 127.11.192.195:41427)
0812 07:07:20.316734 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:20.316739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:20.316757 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:07:20.316870 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:20.316874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:20.316946 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:20.317033 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:20.318336 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:20.319047 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:20.320437 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:20.320871 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:20.320965 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:20.324951 (+ 3986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:20.324963 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:20.324986 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:20.325020 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:20.327593 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:20.328222 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:20.328246 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:20.328248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:20.328402 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:20.328708 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:20.328711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:20.328713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:20.329059 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:20.329270 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:20.329458 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 07:07:20.329608 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 07:07:20.330565 13518 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:20.415670 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 13310
I20260812 07:07:20.440843 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 12712
I20260812 07:07:20.441955 12812 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:20.549203 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 12712
2026-08-12T07:07:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4459 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T07:07:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:20Z Disabled control of system clock
I20260812 07:07:20.639012 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:42563
--webserver_interface=127.11.192.254
--webserver_port=0
--builtin_ntp_servers=127.11.192.212:36653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:42563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:07:20.954629 13613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:20.954881 13613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:20.954998 13613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:20.961828 13613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:20.961920 13613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:20.961982 13613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:20.962056 13613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:20.962122 13613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:20.970674 13613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:36653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:42563
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:42563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:20.971800 13613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:20.973446 13613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:20.984287 13628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:20.984361 13625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:20.984314 13621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:20.985366 13613 server_base.cc:1034] running on GCE node
I20260812 07:07:20.986054 13613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:20.987323 13613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:20.988515 13613 hybrid_clock.cc:648] HybridClock initialized: now 1786518440988460 us; error 49 us; skew 500 ppm
I20260812 07:07:20.988834 13613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:07:20.990983 13613 webserver.cc:460] Webserver started at http://127.11.192.254:35907/ using document root <none> and password file <none>
I20260812 07:07:20.991632 13613 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:20.991731 13613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:20.992030 13613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:20.994837 13613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "914513949b2c4c75b20aa04b1c2acdbd"
format_stamp: "Formatted at 2026-08-12 07:07:20 on dist-test-slave-lgr5"
server_key: "a4075fcb73e261cd9ce36ac9cf5635dd"
server_key_iv: "f35111961afad3fc75f3768a44870ca1"
server_key_version: "encryptionkey@0"
I20260812 07:07:20.995642 13613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "914513949b2c4c75b20aa04b1c2acdbd"
format_stamp: "Formatted at 2026-08-12 07:07:20 on dist-test-slave-lgr5"
server_key: "a4075fcb73e261cd9ce36ac9cf5635dd"
server_key_iv: "f35111961afad3fc75f3768a44870ca1"
server_key_version: "encryptionkey@0"
I20260812 07:07:21.000483 13613 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 07:07:21.003808 13638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:21.005100 13613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:07:21.005261 13613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "914513949b2c4c75b20aa04b1c2acdbd"
format_stamp: "Formatted at 2026-08-12 07:07:20 on dist-test-slave-lgr5"
server_key: "a4075fcb73e261cd9ce36ac9cf5635dd"
server_key_iv: "f35111961afad3fc75f3768a44870ca1"
server_key_version: "encryptionkey@0"
I20260812 07:07:21.005455 13613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:21.036497 13613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:21.037294 13613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:21.037575 13613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:21.054531 13613 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:42563
I20260812 07:07:21.054574 13728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:42563 every 8 connection(s)
I20260812 07:07:21.055804 13613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:21.058493 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 13613
I20260812 07:07:21.058653 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:07:21.059108 12035 external_mini_cluster.cc:1428] Setting key 8e2d75e159c84be7b6c940e3e57c1ff7
I20260812 07:07:21.059391 13729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:21.073581 13733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:21.060896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60208 (local address 127.11.192.254:42563)
0812 07:07:21.061947 (+ 1051us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:21.061961 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:21.062026 (+ 65us) server_negotiation.cc:407] Connection header received
0812 07:07:21.064073 (+ 2047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:21.064133 (+ 60us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:21.064687 (+ 554us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:21.065121 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:21.065634 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:21.067568 (+ 1934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:21.068878 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:21.069855 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:21.070065 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:21.071012 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:21.071023 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:21.071040 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:21.071725 (+ 685us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:21.071821 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:21.071845 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:21.071950 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:21.072118 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 07:07:21.072402 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":343,"threads_started":1}
I20260812 07:07:21.073868 13729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd: Bootstrap starting.
I20260812 07:07:21.076948 13729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:21.077880 13729 log.cc:824] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:21.080641 13729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd: No bootstrap required, opened a new log
I20260812 07:07:21.087733 13729 raft_consensus.cc:348] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "914513949b2c4c75b20aa04b1c2acdbd" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 42563 } }
I20260812 07:07:21.088066 13729 raft_consensus.cc:374] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:21.088207 13729 raft_consensus.cc:729] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 914513949b2c4c75b20aa04b1c2acdbd, State: Initialized, Role: FOLLOWER
I20260812 07:07:21.088698 13729 consensus_queue.cc:260] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "914513949b2c4c75b20aa04b1c2acdbd" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 42563 } }
I20260812 07:07:21.088915 13729 raft_consensus.cc:388] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:21.089035 13729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:21.089184 13729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:21.091190 13729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "914513949b2c4c75b20aa04b1c2acdbd" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 42563 } }
I20260812 07:07:21.091645 13729 leader_election.cc:302] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 914513949b2c4c75b20aa04b1c2acdbd; no voters:
I20260812 07:07:21.092226 13729 leader_election.cc:288] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:21.092319 13737 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:21.092969 13737 raft_consensus.cc:686] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd [term 1 LEADER]: Becoming Leader. State: Replica: 914513949b2c4c75b20aa04b1c2acdbd, State: Running, Role: LEADER
I20260812 07:07:21.093591 13737 consensus_queue.cc:237] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "914513949b2c4c75b20aa04b1c2acdbd" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 42563 } }
I20260812 07:07:21.094249 13729 sys_catalog.cc:564] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:21.096071 13738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 914513949b2c4c75b20aa04b1c2acdbd. Latest consensus state: current_term: 1 leader_uuid: "914513949b2c4c75b20aa04b1c2acdbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "914513949b2c4c75b20aa04b1c2acdbd" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 42563 } } }
I20260812 07:07:21.096544 13738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:21.096134 13739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "914513949b2c4c75b20aa04b1c2acdbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "914513949b2c4c75b20aa04b1c2acdbd" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 42563 } } }
I20260812 07:07:21.096920 13739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:21.098356 13751 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:21.102756 13751 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:21.110630 13751 catalog_manager.cc:1347] Generated new cluster ID: 1c58cd77c1ca4ebcbc08f2fb32411a2f
I20260812 07:07:21.110733 13751 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:21.116783 13751 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:21.118435 13751 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:21.138873 13751 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd: Generated new TSK 0
I20260812 07:07:21.139833 13751 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:07:21.153468 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:42563
--builtin_ntp_servers=127.11.192.212:36653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:07:21.476776 13769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:21.477041 13769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:21.477123 13769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:21.484031 13769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:21.484133 13769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:21.484242 13769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:07:21.493242 13769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:36653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:42563
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:21.494478 13769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:21.496248 13769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:21.508618 13782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:21.508692 13781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:21.508664 13784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:21.509666 13769 server_base.cc:1034] running on GCE node
I20260812 07:07:21.510222 13769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:21.511097 13769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:21.512290 13769 hybrid_clock.cc:648] HybridClock initialized: now 1786518441512232 us; error 47 us; skew 500 ppm
I20260812 07:07:21.512569 13769 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:07:21.514564 13769 webserver.cc:460] Webserver started at http://127.11.192.193:34319/ using document root <none> and password file <none>
I20260812 07:07:21.515199 13769 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:21.515298 13769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:21.515596 13769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:21.518376 13769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "765b64c6b9034d5bbb53cb0e31d8196e"
format_stamp: "Formatted at 2026-08-12 07:07:21 on dist-test-slave-lgr5"
server_key: "b5b96acb78ca1daba77fe7c1b7845833"
server_key_iv: "0c197a080de09415365b7522b87c662d"
server_key_version: "encryptionkey@0"
I20260812 07:07:21.519136 13769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "765b64c6b9034d5bbb53cb0e31d8196e"
format_stamp: "Formatted at 2026-08-12 07:07:21 on dist-test-slave-lgr5"
server_key: "b5b96acb78ca1daba77fe7c1b7845833"
server_key_iv: "0c197a080de09415365b7522b87c662d"
server_key_version: "encryptionkey@0"
I20260812 07:07:21.523929 13769 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 07:07:21.527148 13792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:21.528357 13769 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:07:21.528504 13769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "765b64c6b9034d5bbb53cb0e31d8196e"
format_stamp: "Formatted at 2026-08-12 07:07:21 on dist-test-slave-lgr5"
server_key: "b5b96acb78ca1daba77fe7c1b7845833"
server_key_iv: "0c197a080de09415365b7522b87c662d"
server_key_version: "encryptionkey@0"
I20260812 07:07:21.528703 13769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:21.546139 13769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:21.546838 13769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:21.547119 13769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:21.547999 13769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:21.549337 13769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:21.549431 13769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:21.549532 13769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:21.549602 13769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:21.579316 13769 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:45907
I20260812 07:07:21.579382 13985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:45907 every 8 connection(s)
I20260812 07:07:21.580621 13769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:07:21.582186 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 13769
I20260812 07:07:21.582340 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:07:21.582793 12035 external_mini_cluster.cc:1428] Setting key 9f9340e152e037818d55cdeb9dae7219
I20260812 07:07:21.585605 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:42563
--builtin_ntp_servers=127.11.192.212:36653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:07:21.592703 13988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:21.583024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:43915 (local address 127.11.192.254:42563)
0812 07:07:21.583502 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:21.583508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:21.584753 (+ 1245us) server_negotiation.cc:407] Connection header received
0812 07:07:21.586556 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:21.586562 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:21.586639 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:21.586747 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:21.587669 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:21.588633 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:21.590094 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:21.590612 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:21.590727 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:21.591880 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:21.591886 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:21.591890 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:21.591910 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:21.591955 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:21.591971 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:21.592062 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:21.592284 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 07:07:21.592401 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":231,"threads_started":1}
I20260812 07:07:21.593504 13990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:21.583564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:42563 (local address 127.11.192.193:43915)
0812 07:07:21.584597 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:21.584643 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:21.586309 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:21.586904 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:21.586912 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:21.586989 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:21.587511 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:21.587528 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:21.588763 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:21.588767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:21.589943 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:21.589955 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:21.590783 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:21.590788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:21.590973 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:21.591590 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:07:21.591593 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:21.591628 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:07:21.591703 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:21.592068 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:21.592074 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:21.592100 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 07:07:21.592241 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 07:07:21.592551 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":286,"threads_started":1}
I20260812 07:07:21.594849 13986 heartbeater.cc:347] Connected to a master server at 127.11.192.254:42563
I20260812 07:07:21.595165 13986 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:21.595999 13986 heartbeater.cc:511] Master 127.11.192.254:42563 requested a full tablet report, sending...
I20260812 07:07:21.598117 13669 ts_manager.cc:194] Registered new tserver with Master: 765b64c6b9034d5bbb53cb0e31d8196e (127.11.192.193:45907)
I20260812 07:07:21.599949 13669 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.192.193:43915
W20260812 07:07:21.911365 13991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:21.911633 13991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:21.911775 13991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:21.919106 13991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:21.919209 13991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:21.919392 13991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:07:21.928356 13991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:36653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:42563
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:21.929474 13991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:21.931264 13991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:21.943809 14000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:21.943908 13997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:21.943833 14003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:21.944867 13991 server_base.cc:1034] running on GCE node
I20260812 07:07:21.945434 13991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:21.946326 13991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:21.947548 13991 hybrid_clock.cc:648] HybridClock initialized: now 1786518441947489 us; error 42 us; skew 500 ppm
I20260812 07:07:21.947863 13991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:07:21.950030 13991 webserver.cc:460] Webserver started at http://127.11.192.194:38395/ using document root <none> and password file <none>
I20260812 07:07:21.950645 13991 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:21.950743 13991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:21.951040 13991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:21.953722 13991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "9b8f08f516f74f128aa6191ad03bab95"
format_stamp: "Formatted at 2026-08-12 07:07:21 on dist-test-slave-lgr5"
server_key: "ad0a1a3caa575364e5855de3c1e5e1c9"
server_key_iv: "28b7bd85e57b7af87fda5e6cc738eb6b"
server_key_version: "encryptionkey@0"
I20260812 07:07:21.954499 13991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "9b8f08f516f74f128aa6191ad03bab95"
format_stamp: "Formatted at 2026-08-12 07:07:21 on dist-test-slave-lgr5"
server_key: "ad0a1a3caa575364e5855de3c1e5e1c9"
server_key_iv: "28b7bd85e57b7af87fda5e6cc738eb6b"
server_key_version: "encryptionkey@0"
I20260812 07:07:21.959141 13991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 07:07:21.962306 14012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:21.963496 13991 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:07:21.963641 13991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "9b8f08f516f74f128aa6191ad03bab95"
format_stamp: "Formatted at 2026-08-12 07:07:21 on dist-test-slave-lgr5"
server_key: "ad0a1a3caa575364e5855de3c1e5e1c9"
server_key_iv: "28b7bd85e57b7af87fda5e6cc738eb6b"
server_key_version: "encryptionkey@0"
I20260812 07:07:21.963824 13991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:21.989794 13991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:21.990583 13991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:21.990850 13991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:21.991765 13991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:21.993181 13991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:21.993285 13991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:21.993423 13991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:21.993496 13991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:22.023032 13991 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:40383
I20260812 07:07:22.023093 14210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:40383 every 8 connection(s)
I20260812 07:07:22.024276 13991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:07:22.025751 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 13991
I20260812 07:07:22.025918 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:07:22.026525 12035 external_mini_cluster.cc:1428] Setting key 87203016807d794ecfaf77c9ebcfcbe3
I20260812 07:07:22.029417 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:42563
--builtin_ntp_servers=127.11.192.212:36653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:07:22.036646 13988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:22.026982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:56617 (local address 127.11.192.254:42563)
0812 07:07:22.027195 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:22.027201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:22.028729 (+ 1528us) server_negotiation.cc:407] Connection header received
0812 07:07:22.030542 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:22.030547 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:22.030629 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:22.030717 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:22.031796 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.032585 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.034083 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.034496 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.034604 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:22.035860 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:22.035865 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:22.035868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:22.035884 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:22.035916 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:22.035927 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:22.036001 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:22.036236 (+ 235us) server_negotiation.cc:299] Negotiation successful
0812 07:07:22.036345 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:07:22.037515 14213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:22.027531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:42563 (local address 127.11.192.194:56617)
0812 07:07:22.028576 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:22.028628 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:22.030292 (+ 1664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:22.030953 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:22.030960 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:22.031040 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:22.031636 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:22.031659 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.032754 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.032758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:22.033947 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:22.033959 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.034701 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.034709 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:22.034922 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:22.035557 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:07:22.035560 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:22.035600 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:07:22.035667 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:22.036013 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:22.036025 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:22.036032 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 07:07:22.036214 (+ 182us) client_negotiation.cc:240] Negotiation successful
0812 07:07:22.036560 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":319,"threads_started":1}
I20260812 07:07:22.038854 14211 heartbeater.cc:347] Connected to a master server at 127.11.192.254:42563
I20260812 07:07:22.039105 14211 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:22.039752 14211 heartbeater.cc:511] Master 127.11.192.254:42563 requested a full tablet report, sending...
I20260812 07:07:22.041129 13669 ts_manager.cc:194] Registered new tserver with Master: 9b8f08f516f74f128aa6191ad03bab95 (127.11.192.194:40383)
I20260812 07:07:22.042073 13669 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.192.194:56617
W20260812 07:07:22.356240 14214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:22.356496 14214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:22.356626 14214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:22.363801 14214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:22.363992 14214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:22.364243 14214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:07:22.373705 14214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:36653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:42563
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:22.374876 14214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:22.376664 14214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:22.389433 14226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:22.389470 14222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:22.389523 14221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:22.390368 14214 server_base.cc:1034] running on GCE node
I20260812 07:07:22.390969 14214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:22.391855 14214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:22.393067 14214 hybrid_clock.cc:648] HybridClock initialized: now 1786518442393010 us; error 47 us; skew 500 ppm
I20260812 07:07:22.393392 14214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:07:22.395573 14214 webserver.cc:460] Webserver started at http://127.11.192.195:42415/ using document root <none> and password file <none>
I20260812 07:07:22.396281 14214 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:22.396400 14214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:22.396718 14214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:22.399608 14214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "5fcd077c8d7d463caf920c2559b76383"
format_stamp: "Formatted at 2026-08-12 07:07:22 on dist-test-slave-lgr5"
server_key: "d0f2c9069a3008237f2a40abc1e2797b"
server_key_iv: "187b9a4763944c8f4cee687d70f6fec2"
server_key_version: "encryptionkey@0"
I20260812 07:07:22.400478 14214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "5fcd077c8d7d463caf920c2559b76383"
format_stamp: "Formatted at 2026-08-12 07:07:22 on dist-test-slave-lgr5"
server_key: "d0f2c9069a3008237f2a40abc1e2797b"
server_key_iv: "187b9a4763944c8f4cee687d70f6fec2"
server_key_version: "encryptionkey@0"
I20260812 07:07:22.405483 14214 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 07:07:22.408926 14237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:22.410179 14214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:07:22.410324 14214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "5fcd077c8d7d463caf920c2559b76383"
format_stamp: "Formatted at 2026-08-12 07:07:22 on dist-test-slave-lgr5"
server_key: "d0f2c9069a3008237f2a40abc1e2797b"
server_key_iv: "187b9a4763944c8f4cee687d70f6fec2"
server_key_version: "encryptionkey@0"
I20260812 07:07:22.410521 14214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:22.426319 14214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:22.426998 14214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:22.427256 14214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:22.428170 14214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:22.429477 14214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:22.429582 14214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:22.429682 14214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:22.429749 14214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:22.458775 14214 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:43289
I20260812 07:07:22.458828 14434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:43289 every 8 connection(s)
I20260812 07:07:22.459913 14214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:07:22.468042 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 14214
I20260812 07:07:22.468247 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:07:22.468801 12035 external_mini_cluster.cc:1428] Setting key fad8e32cb01a220955006a81ebc85351
I20260812 07:07:22.471987 13988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:22.462134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:40097 (local address 127.11.192.254:42563)
0812 07:07:22.462370 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:22.462376 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:22.463626 (+ 1250us) server_negotiation.cc:407] Connection header received
0812 07:07:22.465297 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:22.465301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:22.465368 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:22.465456 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:22.466399 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.467159 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.469249 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.469739 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.469856 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:22.471176 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:22.471181 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:22.471184 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:22.471201 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:22.471236 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:22.471248 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:22.471325 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:22.471547 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 07:07:22.471676 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 07:07:22.472833 14437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:22.462640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:42563 (local address 127.11.192.195:40097)
0812 07:07:22.463481 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:22.463515 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:22.465067 (+ 1552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:22.465626 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:22.465633 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:22.465711 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:22.466253 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:22.466276 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.467297 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.467301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:22.469099 (+ 1798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:22.469114 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.469955 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.469963 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:22.470174 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:22.470842 (+ 668us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:07:22.470846 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:22.470891 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:07:22.470979 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:22.471345 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:22.471355 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:22.471363 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 07:07:22.471510 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 07:07:22.471853 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":272,"threads_started":1}
I20260812 07:07:22.473997 14435 heartbeater.cc:347] Connected to a master server at 127.11.192.254:42563
I20260812 07:07:22.474251 14435 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:22.474929 14435 heartbeater.cc:511] Master 127.11.192.254:42563 requested a full tablet report, sending...
I20260812 07:07:22.476238 13669 ts_manager.cc:194] Registered new tserver with Master: 5fcd077c8d7d463caf920c2559b76383 (127.11.192.195:43289)
I20260812 07:07:22.477042 13669 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.192.195:40097
I20260812 07:07:22.483561 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:07:22.492489 13988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:22.487300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60214 (local address 127.11.192.254:42563)
0812 07:07:22.487487 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:22.487492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:22.487871 (+ 379us) server_negotiation.cc:407] Connection header received
0812 07:07:22.487988 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:22.488001 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:22.488058 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:22.488159 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:22.488551 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.489211 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.490455 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.490834 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.490917 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:22.491870 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:22.491874 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:22.491877 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:22.491892 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:22.491918 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:22.491928 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:22.492033 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:22.492167 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 07:07:22.492210 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 07:07:22.495937 13669 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60214:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:07:22.502473 13669 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:07:22.535296 14461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:22.513251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:45907 (local address 127.0.0.1:38622)
0812 07:07:22.526610 (+ 13359us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:22.526649 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:22.526818 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:22.527978 (+ 1160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:22.527985 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:22.528029 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:22.528486 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:22.528499 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.530513 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.530518 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:22.532870 (+ 2352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:22.532885 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.534321 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.534328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:22.534585 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:22.534630 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 07:07:22.534729 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 07:07:22.534899 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12397,"mutex_wait_us":26,"thread_start_us":12582,"threads_started":1}
I20260812 07:07:22.535316 14463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:22.513435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38622 (local address 127.11.192.193:45907)
0812 07:07:22.527355 (+ 13920us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:22.527363 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:22.527394 (+ 31us) server_negotiation.cc:407] Connection header received
0812 07:07:22.527491 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:22.527499 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:22.527747 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:22.528707 (+ 960us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:22.528936 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.530386 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.533174 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.534120 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.534269 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:22.534487 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:22.534809 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 07:07:22.534966 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13765,"thread_start_us":14645,"threads_started":1}
I20260812 07:07:22.537751 14462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:22.513455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:40383 (local address 127.0.0.1:50274)
0812 07:07:22.529143 (+ 15688us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:22.529160 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:22.529263 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:22.530255 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:22.530260 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:22.530284 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:22.530693 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:22.530706 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.532619 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.532624 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:22.535097 (+ 2473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:22.535113 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.536939 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.536945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:22.537117 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:22.537149 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 07:07:22.537256 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 07:07:22.537376 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15564,"mutex_wait_us":110,"thread_start_us":15423,"threads_started":1}
I20260812 07:07:22.537838 14465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:22.514164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50274 (local address 127.11.192.194:40383)
0812 07:07:22.529735 (+ 15571us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:22.529743 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:22.529769 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:07:22.529866 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:22.529873 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:22.530061 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:22.530901 (+ 840us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:22.531164 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.532459 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.535343 (+ 2884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.536739 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.536938 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:22.537134 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:22.537303 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 07:07:22.537455 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15421,"thread_start_us":15292,"threads_started":1}
I20260812 07:07:22.538239 14464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:22.513733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:43289 (local address 127.0.0.1:58476)
0812 07:07:22.530106 (+ 16373us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:22.530119 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:22.530234 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:22.530770 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:22.530774 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:22.530799 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:22.531248 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:22.531261 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.533255 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.533261 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:22.535312 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:22.535324 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.537260 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.537267 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:22.537441 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:22.537471 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 07:07:22.537555 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 07:07:22.537679 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16280,"thread_start_us":16136,"threads_started":1}
I20260812 07:07:22.539439 14468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:22.514237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58476 (local address 127.11.192.195:43289)
0812 07:07:22.528735 (+ 14498us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:22.528742 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:22.530215 (+ 1473us) server_negotiation.cc:407] Connection header received
0812 07:07:22.530374 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:22.530379 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:22.530579 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:22.530766 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:22.531511 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.533098 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.535501 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.536777 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.536944 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:22.537134 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:22.538656 (+ 1522us) server_negotiation.cc:299] Negotiation successful
0812 07:07:22.538815 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14335,"thread_start_us":14173,"threads_started":1}
I20260812 07:07:22.539198 13877 tablet_service.cc:1467] Processing CreateTablet for tablet 94d670df86684e8db4b579ddb3493d3e (DEFAULT_TABLE table=test-table [id=269d40ef72704ec5a79905e665ae4150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:22.540745 13877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94d670df86684e8db4b579ddb3493d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:22.541320 14105 tablet_service.cc:1467] Processing CreateTablet for tablet 94d670df86684e8db4b579ddb3493d3e (DEFAULT_TABLE table=test-table [id=269d40ef72704ec5a79905e665ae4150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:22.542655 14105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94d670df86684e8db4b579ddb3493d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:22.543092 14327 tablet_service.cc:1467] Processing CreateTablet for tablet 94d670df86684e8db4b579ddb3493d3e (DEFAULT_TABLE table=test-table [id=269d40ef72704ec5a79905e665ae4150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:22.544480 14327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94d670df86684e8db4b579ddb3493d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:22.551499 14471 tablet_bootstrap.cc:492] T 94d670df86684e8db4b579ddb3493d3e P 765b64c6b9034d5bbb53cb0e31d8196e: Bootstrap starting.
I20260812 07:07:22.554728 14471 tablet_bootstrap.cc:654] T 94d670df86684e8db4b579ddb3493d3e P 765b64c6b9034d5bbb53cb0e31d8196e: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:22.555773 14471 log.cc:824] T 94d670df86684e8db4b579ddb3493d3e P 765b64c6b9034d5bbb53cb0e31d8196e: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:22.556463 14472 tablet_bootstrap.cc:492] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95: Bootstrap starting.
I20260812 07:07:22.558571 14481 tablet_bootstrap.cc:492] T 94d670df86684e8db4b579ddb3493d3e P 5fcd077c8d7d463caf920c2559b76383: Bootstrap starting.
I20260812 07:07:22.559724 14471 tablet_bootstrap.cc:492] T 94d670df86684e8db4b579ddb3493d3e P 765b64c6b9034d5bbb53cb0e31d8196e: No bootstrap required, opened a new log
I20260812 07:07:22.559891 14472 tablet_bootstrap.cc:654] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:22.560017 14471 ts_tablet_manager.cc:1397] T 94d670df86684e8db4b579ddb3493d3e P 765b64c6b9034d5bbb53cb0e31d8196e: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260812 07:07:22.561158 14481 tablet_bootstrap.cc:654] T 94d670df86684e8db4b579ddb3493d3e P 5fcd077c8d7d463caf920c2559b76383: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:22.561134 14472 log.cc:824] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:22.562227 14481 log.cc:824] T 94d670df86684e8db4b579ddb3493d3e P 5fcd077c8d7d463caf920c2559b76383: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:22.563901 14472 tablet_bootstrap.cc:492] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95: No bootstrap required, opened a new log
I20260812 07:07:22.564196 14472 ts_tablet_manager.cc:1397] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260812 07:07:22.565187 14481 tablet_bootstrap.cc:492] T 94d670df86684e8db4b579ddb3493d3e P 5fcd077c8d7d463caf920c2559b76383: No bootstrap required, opened a new log
I20260812 07:07:22.565428 14481 ts_tablet_manager.cc:1397] T 94d670df86684e8db4b579ddb3493d3e P 5fcd077c8d7d463caf920c2559b76383: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.004s
I20260812 07:07:22.568128 14471 raft_consensus.cc:348] T 94d670df86684e8db4b579ddb3493d3e P 765b64c6b9034d5bbb53cb0e31d8196e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765b64c6b9034d5bbb53cb0e31d8196e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45907 } } peers { permanent_uuid: "9b8f08f516f74f128aa6191ad03bab95" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 40383 } } peers { permanent_uuid: "5fcd077c8d7d463caf920c2559b76383" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 43289 } }
I20260812 07:07:22.568496 14471 raft_consensus.cc:374] T 94d670df86684e8db4b579ddb3493d3e P 765b64c6b9034d5bbb53cb0e31d8196e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:22.568593 14471 raft_consensus.cc:729] T 94d670df86684e8db4b579ddb3493d3e P 765b64c6b9034d5bbb53cb0e31d8196e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 765b64c6b9034d5bbb53cb0e31d8196e, State: Initialized, Role: FOLLOWER
I20260812 07:07:22.569139 14471 consensus_queue.cc:260] T 94d670df86684e8db4b579ddb3493d3e P 765b64c6b9034d5bbb53cb0e31d8196e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765b64c6b9034d5bbb53cb0e31d8196e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45907 } } peers { permanent_uuid: "9b8f08f516f74f128aa6191ad03bab95" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 40383 } } peers { permanent_uuid: "5fcd077c8d7d463caf920c2559b76383" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 43289 } }
I20260812 07:07:22.570106 13986 heartbeater.cc:503] Master 127.11.192.254:42563 was elected leader, sending a full tablet report...
I20260812 07:07:22.570736 14471 ts_tablet_manager.cc:1428] T 94d670df86684e8db4b579ddb3493d3e P 765b64c6b9034d5bbb53cb0e31d8196e: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260812 07:07:22.574368 14472 raft_consensus.cc:348] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765b64c6b9034d5bbb53cb0e31d8196e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45907 } } peers { permanent_uuid: "9b8f08f516f74f128aa6191ad03bab95" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 40383 } } peers { permanent_uuid: "5fcd077c8d7d463caf920c2559b76383" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 43289 } }
I20260812 07:07:22.574769 14472 raft_consensus.cc:374] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:22.574869 14472 raft_consensus.cc:729] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b8f08f516f74f128aa6191ad03bab95, State: Initialized, Role: FOLLOWER
I20260812 07:07:22.575309 14481 raft_consensus.cc:348] T 94d670df86684e8db4b579ddb3493d3e P 5fcd077c8d7d463caf920c2559b76383 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765b64c6b9034d5bbb53cb0e31d8196e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45907 } } peers { permanent_uuid: "9b8f08f516f74f128aa6191ad03bab95" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 40383 } } peers { permanent_uuid: "5fcd077c8d7d463caf920c2559b76383" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 43289 } }
I20260812 07:07:22.575446 14472 consensus_queue.cc:260] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765b64c6b9034d5bbb53cb0e31d8196e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45907 } } peers { permanent_uuid: "9b8f08f516f74f128aa6191ad03bab95" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 40383 } } peers { permanent_uuid: "5fcd077c8d7d463caf920c2559b76383" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 43289 } }
I20260812 07:07:22.575711 14481 raft_consensus.cc:374] T 94d670df86684e8db4b579ddb3493d3e P 5fcd077c8d7d463caf920c2559b76383 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:22.575807 14481 raft_consensus.cc:729] T 94d670df86684e8db4b579ddb3493d3e P 5fcd077c8d7d463caf920c2559b76383 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fcd077c8d7d463caf920c2559b76383, State: Initialized, Role: FOLLOWER
I20260812 07:07:22.576695 14211 heartbeater.cc:503] Master 127.11.192.254:42563 was elected leader, sending a full tablet report...
I20260812 07:07:22.577324 14472 ts_tablet_manager.cc:1428] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260812 07:07:22.576399 14481 consensus_queue.cc:260] T 94d670df86684e8db4b579ddb3493d3e P 5fcd077c8d7d463caf920c2559b76383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765b64c6b9034d5bbb53cb0e31d8196e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45907 } } peers { permanent_uuid: "9b8f08f516f74f128aa6191ad03bab95" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 40383 } } peers { permanent_uuid: "5fcd077c8d7d463caf920c2559b76383" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 43289 } }
I20260812 07:07:22.579183 14481 ts_tablet_manager.cc:1428] T 94d670df86684e8db4b579ddb3493d3e P 5fcd077c8d7d463caf920c2559b76383: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
I20260812 07:07:22.579525 14435 heartbeater.cc:503] Master 127.11.192.254:42563 was elected leader, sending a full tablet report...
W20260812 07:07:22.582553 13987 tablet.cc:2369] T 94d670df86684e8db4b579ddb3493d3e P 765b64c6b9034d5bbb53cb0e31d8196e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:07:22.606644 14488 raft_consensus.cc:482] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:07:22.606891 14488 raft_consensus.cc:504] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765b64c6b9034d5bbb53cb0e31d8196e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45907 } } peers { permanent_uuid: "9b8f08f516f74f128aa6191ad03bab95" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 40383 } } peers { permanent_uuid: "5fcd077c8d7d463caf920c2559b76383" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 43289 } }
I20260812 07:07:22.608373 14488 leader_election.cc:288] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 765b64c6b9034d5bbb53cb0e31d8196e (127.11.192.193:45907), 5fcd077c8d7d463caf920c2559b76383 (127.11.192.195:43289)
I20260812 07:07:22.616110 14468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:22.608953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:41059 (local address 127.11.192.195:43289)
0812 07:07:22.609116 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:22.609121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:22.609757 (+ 636us) server_negotiation.cc:407] Connection header received
0812 07:07:22.609945 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:22.609950 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:22.610040 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:22.610137 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:22.610844 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.611894 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.614273 (+ 2379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.615180 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.615334 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:22.615401 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:22.615743 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 07:07:22.615863 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:07:22.616146 14463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:22.608871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:51211 (local address 127.11.192.193:45907)
0812 07:07:22.609061 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:22.609067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:22.609756 (+ 689us) server_negotiation.cc:407] Connection header received
0812 07:07:22.609969 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:22.609974 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:22.610050 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:22.610136 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:22.610844 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.611898 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.614295 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.615142 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.615295 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:22.615367 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:22.615768 (+ 401us) server_negotiation.cc:299] Negotiation successful
0812 07:07:22.615890 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:07:22.616217 14491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:22.608578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:43289 (local address 127.11.192.194:41059)
0812 07:07:22.609645 (+ 1067us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:22.609664 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:22.609804 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:22.610210 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:22.610214 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:22.610236 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:22.610731 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:22.610741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.612019 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.612023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:22.614069 (+ 2046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:22.614085 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.615387 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.615394 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:22.615577 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:22.615618 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 07:07:22.615689 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 07:07:22.615810 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":885,"mutex_wait_us":159,"thread_start_us":697,"threads_started":1}
I20260812 07:07:22.616429 14490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:22.608593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:45907 (local address 127.11.192.194:51211)
0812 07:07:22.609645 (+ 1052us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:22.609664 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:22.609853 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:22.610158 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:22.610163 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:22.610189 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:22.610686 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:22.610701 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.612017 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.612022 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:22.614090 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:22.614105 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.615367 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.615374 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:22.615610 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:22.615634 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 07:07:22.615719 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 07:07:22.615844 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":957,"mutex_wait_us":114,"spinlock_wait_cycles":59776,"thread_start_us":665,"threads_started":1}
I20260812 07:07:22.617370 13909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94d670df86684e8db4b579ddb3493d3e" candidate_uuid: "9b8f08f516f74f128aa6191ad03bab95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "765b64c6b9034d5bbb53cb0e31d8196e" is_pre_election: true
I20260812 07:07:22.617370 14358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94d670df86684e8db4b579ddb3493d3e" candidate_uuid: "9b8f08f516f74f128aa6191ad03bab95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fcd077c8d7d463caf920c2559b76383" is_pre_election: true
I20260812 07:07:22.617908 13909 raft_consensus.cc:2454] T 94d670df86684e8db4b579ddb3493d3e P 765b64c6b9034d5bbb53cb0e31d8196e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b8f08f516f74f128aa6191ad03bab95 in term 0.
I20260812 07:07:22.617908 14358 raft_consensus.cc:2454] T 94d670df86684e8db4b579ddb3493d3e P 5fcd077c8d7d463caf920c2559b76383 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b8f08f516f74f128aa6191ad03bab95 in term 0.
I20260812 07:07:22.618539 14020 leader_election.cc:302] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fcd077c8d7d463caf920c2559b76383, 9b8f08f516f74f128aa6191ad03bab95; no voters:
I20260812 07:07:22.618858 14488 raft_consensus.cc:2781] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:07:22.619005 14488 raft_consensus.cc:482] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:07:22.619119 14488 raft_consensus.cc:3037] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:22.621229 14488 raft_consensus.cc:504] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765b64c6b9034d5bbb53cb0e31d8196e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45907 } } peers { permanent_uuid: "9b8f08f516f74f128aa6191ad03bab95" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 40383 } } peers { permanent_uuid: "5fcd077c8d7d463caf920c2559b76383" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 43289 } }
I20260812 07:07:22.621784 14488 leader_election.cc:288] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95 [CANDIDATE]: Term 1 election: Requested vote from peers 765b64c6b9034d5bbb53cb0e31d8196e (127.11.192.193:45907), 5fcd077c8d7d463caf920c2559b76383 (127.11.192.195:43289)
I20260812 07:07:22.622125 13909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94d670df86684e8db4b579ddb3493d3e" candidate_uuid: "9b8f08f516f74f128aa6191ad03bab95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "765b64c6b9034d5bbb53cb0e31d8196e"
I20260812 07:07:22.622280 14358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94d670df86684e8db4b579ddb3493d3e" candidate_uuid: "9b8f08f516f74f128aa6191ad03bab95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fcd077c8d7d463caf920c2559b76383"
I20260812 07:07:22.622314 13909 raft_consensus.cc:3037] T 94d670df86684e8db4b579ddb3493d3e P 765b64c6b9034d5bbb53cb0e31d8196e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:22.622464 14358 raft_consensus.cc:3037] T 94d670df86684e8db4b579ddb3493d3e P 5fcd077c8d7d463caf920c2559b76383 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:22.625404 13909 raft_consensus.cc:2454] T 94d670df86684e8db4b579ddb3493d3e P 765b64c6b9034d5bbb53cb0e31d8196e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b8f08f516f74f128aa6191ad03bab95 in term 1.
I20260812 07:07:22.625432 14358 raft_consensus.cc:2454] T 94d670df86684e8db4b579ddb3493d3e P 5fcd077c8d7d463caf920c2559b76383 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b8f08f516f74f128aa6191ad03bab95 in term 1.
I20260812 07:07:22.625808 14020 leader_election.cc:302] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5fcd077c8d7d463caf920c2559b76383, 765b64c6b9034d5bbb53cb0e31d8196e, 9b8f08f516f74f128aa6191ad03bab95; no voters:
I20260812 07:07:22.626097 14488 raft_consensus.cc:2781] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:22.626482 14488 raft_consensus.cc:686] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95 [term 1 LEADER]: Becoming Leader. State: Replica: 9b8f08f516f74f128aa6191ad03bab95, State: Running, Role: LEADER
I20260812 07:07:22.626936 14488 consensus_queue.cc:237] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765b64c6b9034d5bbb53cb0e31d8196e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45907 } } peers { permanent_uuid: "9b8f08f516f74f128aa6191ad03bab95" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 40383 } } peers { permanent_uuid: "5fcd077c8d7d463caf920c2559b76383" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 43289 } }
I20260812 07:07:22.630901 13668 catalog_manager.cc:5495] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b8f08f516f74f128aa6191ad03bab95 (127.11.192.194). New cstate: current_term: 1 leader_uuid: "9b8f08f516f74f128aa6191ad03bab95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "765b64c6b9034d5bbb53cb0e31d8196e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b8f08f516f74f128aa6191ad03bab95" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 40383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fcd077c8d7d463caf920c2559b76383" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 43289 } health_report { overall_health: UNKNOWN } } }
I20260812 07:07:22.647567 14465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:22.642458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50286 (local address 127.11.192.194:40383)
0812 07:07:22.642645 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:22.642650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:22.642727 (+ 77us) server_negotiation.cc:407] Connection header received
0812 07:07:22.642918 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:22.642922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:22.642991 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:22.643112 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:07:22.643472 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.644185 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.645547 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.645927 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.646032 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:22.646594 (+ 562us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:07:22.647023 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:07:22.647170 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:22.647322 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 07:07:22.647361 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:07:22.654673 13909 raft_consensus.cc:1261] T 94d670df86684e8db4b579ddb3493d3e P 765b64c6b9034d5bbb53cb0e31d8196e [term 1 FOLLOWER]: Refusing update from remote peer 9b8f08f516f74f128aa6191ad03bab95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:07:22.654672 14358 raft_consensus.cc:1261] T 94d670df86684e8db4b579ddb3493d3e P 5fcd077c8d7d463caf920c2559b76383 [term 1 FOLLOWER]: Refusing update from remote peer 9b8f08f516f74f128aa6191ad03bab95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:07:22.655534 14492 consensus_queue.cc:1035] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fcd077c8d7d463caf920c2559b76383" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 43289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:07:22.655818 14488 consensus_queue.cc:1035] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "765b64c6b9034d5bbb53cb0e31d8196e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:07:22.680315 13668 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:07:22.680593 13668 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:07:22.684051 13668 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 914513949b2c4c75b20aa04b1c2acdbd: Sending DeleteTablet for 3 replicas of tablet 94d670df86684e8db4b579ddb3493d3e
I20260812 07:07:22.685086 13877 tablet_service.cc:1514] Processing DeleteTablet for tablet 94d670df86684e8db4b579ddb3493d3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:22 UTC) from {username='slave'} at 127.0.0.1:38622
I20260812 07:07:22.685178 14105 tablet_service.cc:1514] Processing DeleteTablet for tablet 94d670df86684e8db4b579ddb3493d3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:22 UTC) from {username='slave'} at 127.0.0.1:50274
I20260812 07:07:22.685832 14327 tablet_service.cc:1514] Processing DeleteTablet for tablet 94d670df86684e8db4b579ddb3493d3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:22 UTC) from {username='slave'} at 127.0.0.1:58476
I20260812 07:07:22.686290 14517 tablet_replica.cc:330] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95: stopping tablet replica
I20260812 07:07:22.686703 14517 raft_consensus.cc:2229] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:07:22.686738 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 13769
I20260812 07:07:22.687299 14517 raft_consensus.cc:2258] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:22.687995 14515 tablet_replica.cc:330] T 94d670df86684e8db4b579ddb3493d3e P 765b64c6b9034d5bbb53cb0e31d8196e: stopping tablet replica
I20260812 07:07:22.688314 14520 tablet_replica.cc:330] T 94d670df86684e8db4b579ddb3493d3e P 5fcd077c8d7d463caf920c2559b76383: stopping tablet replica
I20260812 07:07:22.688453 14515 raft_consensus.cc:2229] T 94d670df86684e8db4b579ddb3493d3e P 765b64c6b9034d5bbb53cb0e31d8196e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:07:22.688938 14515 raft_consensus.cc:2258] T 94d670df86684e8db4b579ddb3493d3e P 765b64c6b9034d5bbb53cb0e31d8196e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:22.688897 14520 raft_consensus.cc:2229] T 94d670df86684e8db4b579ddb3493d3e P 5fcd077c8d7d463caf920c2559b76383 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:07:22.689396 14520 raft_consensus.cc:2258] T 94d670df86684e8db4b579ddb3493d3e P 5fcd077c8d7d463caf920c2559b76383 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:22.691360 14515 ts_tablet_manager.cc:1905] T 94d670df86684e8db4b579ddb3493d3e P 765b64c6b9034d5bbb53cb0e31d8196e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:22.695638 14517 ts_tablet_manager.cc:1905] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:22.698566 14520 ts_tablet_manager.cc:1905] T 94d670df86684e8db4b579ddb3493d3e P 5fcd077c8d7d463caf920c2559b76383: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:22.698647 14515 ts_tablet_manager.cc:1918] T 94d670df86684e8db4b579ddb3493d3e P 765b64c6b9034d5bbb53cb0e31d8196e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:07:22.698849 14515 log.cc:1196] T 94d670df86684e8db4b579ddb3493d3e P 765b64c6b9034d5bbb53cb0e31d8196e: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-0/wal/wals/94d670df86684e8db4b579ddb3493d3e
I20260812 07:07:22.699687 14515 ts_tablet_manager.cc:1939] T 94d670df86684e8db4b579ddb3493d3e P 765b64c6b9034d5bbb53cb0e31d8196e: Deleting consensus metadata
I20260812 07:07:22.701486 13639 catalog_manager.cc:4841] TS 765b64c6b9034d5bbb53cb0e31d8196e (127.11.192.193:45907): tablet 94d670df86684e8db4b579ddb3493d3e (table test-table [id=269d40ef72704ec5a79905e665ae4150]) successfully deleted
I20260812 07:07:22.702770 14517 ts_tablet_manager.cc:1918] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:07:22.702987 14517 log.cc:1196] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/94d670df86684e8db4b579ddb3493d3e
I20260812 07:07:22.703583 14517 ts_tablet_manager.cc:1939] T 94d670df86684e8db4b579ddb3493d3e P 9b8f08f516f74f128aa6191ad03bab95: Deleting consensus metadata
I20260812 07:07:22.704152 14463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:22.688383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38636 (local address 127.11.192.193:45907)
0812 07:07:22.688553 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:22.688559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:22.694564 (+ 6005us) server_negotiation.cc:407] Connection header received
0812 07:07:22.694795 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:22.694800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:22.694893 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:22.695002 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:22.695657 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.696853 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.700247 (+ 3394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.700917 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.701032 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:22.702501 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:22.702512 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:22.702528 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:22.703226 (+ 698us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:22.703285 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:22.703311 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:22.703395 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:22.703648 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 07:07:22.703759 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 07:07:22.705405 13972 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:22.705577 13640 catalog_manager.cc:4841] TS 9b8f08f516f74f128aa6191ad03bab95 (127.11.192.194:40383): tablet 94d670df86684e8db4b579ddb3493d3e (table test-table [id=269d40ef72704ec5a79905e665ae4150]) successfully deleted
I20260812 07:07:22.707481 14520 ts_tablet_manager.cc:1918] T 94d670df86684e8db4b579ddb3493d3e P 5fcd077c8d7d463caf920c2559b76383: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:07:22.707677 14520 log.cc:1196] T 94d670df86684e8db4b579ddb3493d3e P 5fcd077c8d7d463caf920c2559b76383: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/94d670df86684e8db4b579ddb3493d3e
I20260812 07:07:22.708297 14520 ts_tablet_manager.cc:1939] T 94d670df86684e8db4b579ddb3493d3e P 5fcd077c8d7d463caf920c2559b76383: Deleting consensus metadata
I20260812 07:07:22.709867 13647 catalog_manager.cc:4841] TS 5fcd077c8d7d463caf920c2559b76383 (127.11.192.195:43289): tablet 94d670df86684e8db4b579ddb3493d3e (table test-table [id=269d40ef72704ec5a79905e665ae4150]) successfully deleted
I20260812 07:07:22.791289 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 13769
I20260812 07:07:22.813652 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 13991
I20260812 07:07:22.820783 14465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:22.814410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50300 (local address 127.11.192.194:40383)
0812 07:07:22.814600 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:22.814609 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:22.814633 (+ 24us) server_negotiation.cc:407] Connection header received
0812 07:07:22.814847 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:22.814853 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:22.814958 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:22.815077 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:22.815526 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.816379 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.817606 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.818022 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.818139 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:22.819231 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:22.819240 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:22.819255 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:22.819872 (+ 617us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:22.819923 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:22.819944 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:22.820048 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:22.820338 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 07:07:22.820444 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:07:22.821468 14205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:22.929488 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 13991
I20260812 07:07:22.957619 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 14214
I20260812 07:07:22.964303 14468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:22.958346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58478 (local address 127.11.192.195:43289)
0812 07:07:22.958527 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:22.958532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:22.958591 (+ 59us) server_negotiation.cc:407] Connection header received
0812 07:07:22.958779 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:22.958785 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:22.958886 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:22.958985 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:22.959384 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.960163 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.961386 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:22.961794 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:22.961906 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:22.962932 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:22.962941 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:22.962956 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:22.963532 (+ 576us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:22.963581 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:22.963607 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:22.963709 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:22.963845 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 07:07:22.963953 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:07:22.964900 14430 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:23.051393 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 14214
I20260812 07:07:23.078764 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 13613
I20260812 07:07:23.079782 13724 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:23.186439 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 13613
2026-08-12T07:07:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2637 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:07:23 dist-test-slave-lgr5 krb5kdc[14536](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:07:23 dist-test-slave-lgr5 krb5kdc[14536](info): set up 2 sockets
Aug 12 07:07:23 dist-test-slave-lgr5 krb5kdc[14536](info): commencing operation
krb5kdc: starting...
W20260812 07:07:24.453728 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:07:24 dist-test-slave-lgr5 krb5kdc[14536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:07:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:07:24.712450 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:36623
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:34207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:36623
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:25.027736 14561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:25.027987 14561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:25.028133 14561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:25.034976 14561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:25.035089 14561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:25.035166 14561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:25.035236 14561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:25.035296 14561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:25.044507 14561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:34207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:36623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:36623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:25.045621 14561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:25.046824 14561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:25.057549 14571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:25.057699 14569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:25.057658 14568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:25.058501 14561 server_base.cc:1034] running on GCE node
I20260812 07:07:25.059191 14561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:25.060506 14561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:25.061702 14561 hybrid_clock.cc:648] HybridClock initialized: now 1786518445061650 us; error 42 us; skew 500 ppm
I20260812 07:07:25.062018 14561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:25 dist-test-slave-lgr5 krb5kdc[14536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518445, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:25.064841 14561 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:07:25.066237 14561 webserver.cc:460] Webserver started at http://127.11.192.254:42637/ using document root <none> and password file <none>
I20260812 07:07:25.066845 14561 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:25.066958 14561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:25.067239 14561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:25.070050 14561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "b518c8d0f98448d4b87f08807a3f3f0a"
format_stamp: "Formatted at 2026-08-12 07:07:25 on dist-test-slave-lgr5"
server_key: "f9c17b44ba7870ca584af2a7a1136ff8"
server_key_iv: "d5ae1d348817e277bb1101010e0e72c6"
server_key_version: "encryptionkey@0"
I20260812 07:07:25.070752 14561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "b518c8d0f98448d4b87f08807a3f3f0a"
format_stamp: "Formatted at 2026-08-12 07:07:25 on dist-test-slave-lgr5"
server_key: "f9c17b44ba7870ca584af2a7a1136ff8"
server_key_iv: "d5ae1d348817e277bb1101010e0e72c6"
server_key_version: "encryptionkey@0"
I20260812 07:07:25.075531 14561 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260812 07:07:25.078704 14580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:25.079873 14561 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:07:25.080019 14561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "b518c8d0f98448d4b87f08807a3f3f0a"
format_stamp: "Formatted at 2026-08-12 07:07:25 on dist-test-slave-lgr5"
server_key: "f9c17b44ba7870ca584af2a7a1136ff8"
server_key_iv: "d5ae1d348817e277bb1101010e0e72c6"
server_key_version: "encryptionkey@0"
I20260812 07:07:25.080205 14561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:25.092563 14561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:25.096015 14561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:25.096272 14561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:25.112850 14561 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:36623
I20260812 07:07:25.112879 14672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:36623 every 8 connection(s)
I20260812 07:07:25.114051 14561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:25.117527 14673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:25.122990 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 14561
I20260812 07:07:25.123178 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:07:25.123744 12035 external_mini_cluster.cc:1428] Setting key d3eb516e90525ae07260d88d8b3945d2
I20260812 07:07:25.128999 14673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a: Bootstrap starting.
I20260812 07:07:25.132594 14673 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:25.133718 14673 log.cc:824] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:25.136608 14673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a: No bootstrap required, opened a new log
Aug 12 07:07:25 dist-test-slave-lgr5 krb5kdc[14536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:07:25.145061 14673 raft_consensus.cc:348] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b518c8d0f98448d4b87f08807a3f3f0a" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36623 } }
I20260812 07:07:25.145466 14673 raft_consensus.cc:374] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:25.145573 14673 raft_consensus.cc:729] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b518c8d0f98448d4b87f08807a3f3f0a, State: Initialized, Role: FOLLOWER
I20260812 07:07:25.146202 14673 consensus_queue.cc:260] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b518c8d0f98448d4b87f08807a3f3f0a" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36623 } }
I20260812 07:07:25.146431 14673 raft_consensus.cc:388] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:25.146574 14673 raft_consensus.cc:482] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:25.146730 14673 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:25.147359 14676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:25.125659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46358 (local address 127.11.192.254:36623)
0812 07:07:25.126388 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:25.126398 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:25.126440 (+ 42us) server_negotiation.cc:407] Connection header received
0812 07:07:25.128760 (+ 2320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:25.128788 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:25.129220 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:25.129638 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:25.131387 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:25.133010 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:25.134683 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:25.135389 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:25.135524 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:25.140439 (+ 4915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:25.140449 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:25.140466 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:25.140504 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:25.143358 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:25.144134 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:25.144140 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:25.144146 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:25.144316 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:25.144691 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:25.144694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:25.144696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:25.145151 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:25.145364 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:25.145882 (+ 518us) server_negotiation.cc:299] Negotiation successful
0812 07:07:25.146291 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":343,"threads_started":1}
I20260812 07:07:25.149399 14673 raft_consensus.cc:504] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b518c8d0f98448d4b87f08807a3f3f0a" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36623 } }
I20260812 07:07:25.149917 14673 leader_election.cc:302] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b518c8d0f98448d4b87f08807a3f3f0a; no voters:
I20260812 07:07:25.150435 14673 leader_election.cc:288] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:25.150538 14681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:25.151198 14681 raft_consensus.cc:686] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a [term 1 LEADER]: Becoming Leader. State: Replica: b518c8d0f98448d4b87f08807a3f3f0a, State: Running, Role: LEADER
I20260812 07:07:25.151712 14681 consensus_queue.cc:237] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b518c8d0f98448d4b87f08807a3f3f0a" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36623 } }
I20260812 07:07:25.152271 14673 sys_catalog.cc:564] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:25.154361 14683 sys_catalog.cc:455] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b518c8d0f98448d4b87f08807a3f3f0a. Latest consensus state: current_term: 1 leader_uuid: "b518c8d0f98448d4b87f08807a3f3f0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b518c8d0f98448d4b87f08807a3f3f0a" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36623 } } }
I20260812 07:07:25.154448 14682 sys_catalog.cc:455] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b518c8d0f98448d4b87f08807a3f3f0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b518c8d0f98448d4b87f08807a3f3f0a" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 36623 } } }
I20260812 07:07:25.154757 14683 sys_catalog.cc:458] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:25.154757 14682 sys_catalog.cc:458] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:25.159977 14695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:25.165001 14695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:25.173105 14695 catalog_manager.cc:1347] Generated new cluster ID: 6c468a2522a24f5c8d10a860290bff31
I20260812 07:07:25.173209 14695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:25.204917 14695 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:25.206576 14695 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:25.216791 14695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a: Generated new TSK 0
I20260812 07:07:25.217705 14695 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
Entry for principal kudu/127.11.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab.
I20260812 07:07:25.290930 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36623
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:34207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:25.614982 14722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:25.615242 14722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:25.615367 14722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:25.622237 14722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:25.622326 14722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:25.622469 14722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:07:25.631501 14722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:34207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.193.keytab
--principal=kudu/127.11.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:25.632593 14722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:25.633790 14722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:25.645948 14729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:25.645876 14736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:25.645913 14738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:25.646796 14722 server_base.cc:1034] running on GCE node
I20260812 07:07:25.647313 14722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:25.648180 14722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:25.649371 14722 hybrid_clock.cc:648] HybridClock initialized: now 1786518445649311 us; error 54 us; skew 500 ppm
I20260812 07:07:25.649660 14722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:25 dist-test-slave-lgr5 krb5kdc[14536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518445, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:25.652593 14722 init.cc:377] Logged in from keytab as kudu/127.11.192.193@KRBTEST.COM (short username kudu)
I20260812 07:07:25.653868 14722 webserver.cc:460] Webserver started at http://127.11.192.193:44203/ using document root <none> and password file <none>
I20260812 07:07:25.654475 14722 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:25.654590 14722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:25.654883 14722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:25.657624 14722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "48ba28d907bd414ebccdd5f39e6aefd7"
format_stamp: "Formatted at 2026-08-12 07:07:25 on dist-test-slave-lgr5"
server_key: "f50d788df3cacd380b2639c01917e2af"
server_key_iv: "33d0460c88cda747825d1c8eaf6eb917"
server_key_version: "encryptionkey@0"
I20260812 07:07:25.658335 14722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "48ba28d907bd414ebccdd5f39e6aefd7"
format_stamp: "Formatted at 2026-08-12 07:07:25 on dist-test-slave-lgr5"
server_key: "f50d788df3cacd380b2639c01917e2af"
server_key_iv: "33d0460c88cda747825d1c8eaf6eb917"
server_key_version: "encryptionkey@0"
I20260812 07:07:25.663103 14722 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:07:25.666302 14751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:25.667488 14722 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:07:25.667635 14722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "48ba28d907bd414ebccdd5f39e6aefd7"
format_stamp: "Formatted at 2026-08-12 07:07:25 on dist-test-slave-lgr5"
server_key: "f50d788df3cacd380b2639c01917e2af"
server_key_iv: "33d0460c88cda747825d1c8eaf6eb917"
server_key_version: "encryptionkey@0"
I20260812 07:07:25.667814 14722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:25.696660 14722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:25.700107 14722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:25.700412 14722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:25.701534 14722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:25.703058 14722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:25.703148 14722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:25.703261 14722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:25.703325 14722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:25.732693 14722 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:40809
I20260812 07:07:25.732759 14950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:40809 every 8 connection(s)
I20260812 07:07:25.733858 14722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:07:25.735306 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 14722
I20260812 07:07:25.735476 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:07:25.736020 12035 external_mini_cluster.cc:1428] Setting key df2752a7d9e0e712210c13ea333dc885
Aug 12 07:07:25 dist-test-slave-lgr5 krb5kdc[14536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518445, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.193@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:07:25.752661 14956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:25.736337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:50639 (local address 127.11.192.254:36623)
0812 07:07:25.736948 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:25.736955 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:25.737875 (+ 920us) server_negotiation.cc:407] Connection header received
0812 07:07:25.739821 (+ 1946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:25.739826 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:25.739901 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:25.739994 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:25.741456 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:25.742180 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:25.743677 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:25.744065 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:25.744209 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:25.747493 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:25.747498 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:25.747501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:25.747531 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:25.749799 (+ 2268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:25.750452 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:25.750456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:25.750458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:25.750573 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:25.750983 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:25.750987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:25.750989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:25.751230 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:25.751376 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:25.752040 (+ 664us) server_negotiation.cc:299] Negotiation successful
0812 07:07:25.752281 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":369,"threads_started":1}
I20260812 07:07:25.753546 14958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:25.737037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:36623 (local address 127.11.192.193:50639)
0812 07:07:25.737747 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:25.737786 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:25.739574 (+ 1788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:25.740273 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:25.740280 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:25.740756 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:25.741294 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:25.741311 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:25.742337 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:25.742341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:25.743540 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:25.743553 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:25.744299 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:25.744306 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:25.744538 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:25.745143 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:25.745164 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:25.747294 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:25.749934 (+ 2640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:25.749942 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:25.749956 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:25.750340 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:25.750673 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:25.750679 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:25.750683 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:25.750867 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:25.751355 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:25.751362 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:25.751681 (+ 319us) client_negotiation.cc:769] Sending connection context
0812 07:07:25.751903 (+ 222us) client_negotiation.cc:240] Negotiation successful
0812 07:07:25.752401 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":299,"threads_started":1}
I20260812 07:07:25.754995 14952 heartbeater.cc:347] Connected to a master server at 127.11.192.254:36623
I20260812 07:07:25.755263 14952 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:25.755950 14952 heartbeater.cc:511] Master 127.11.192.254:36623 requested a full tablet report, sending...
I20260812 07:07:25.758298 14606 ts_manager.cc:194] Registered new tserver with Master: 48ba28d907bd414ebccdd5f39e6aefd7 (127.11.192.193:40809)
I20260812 07:07:25.760227 14606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.193@KRBTEST.COM'} at 127.11.192.193:50639
WARNING: no policy specified for kudu/127.11.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
Entry for principal kudu/127.11.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab.
I20260812 07:07:25.796468 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36623
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:34207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:26.112310 14963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:26.112605 14963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:26.112742 14963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:26.119740 14963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:26.119829 14963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:26.119966 14963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:07:26.129240 14963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:34207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.194.keytab
--principal=kudu/127.11.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:26.130317 14963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:26.131508 14963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:26.144202 14973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:26.144280 14970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:26.144258 14971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:26.145099 14963 server_base.cc:1034] running on GCE node
I20260812 07:07:26.145721 14963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:26.146600 14963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:26.147789 14963 hybrid_clock.cc:648] HybridClock initialized: now 1786518446147739 us; error 44 us; skew 500 ppm
I20260812 07:07:26.148068 14963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:26 dist-test-slave-lgr5 krb5kdc[14536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518446, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:26.151013 14963 init.cc:377] Logged in from keytab as kudu/127.11.192.194@KRBTEST.COM (short username kudu)
I20260812 07:07:26.152495 14963 webserver.cc:460] Webserver started at http://127.11.192.194:41187/ using document root <none> and password file <none>
I20260812 07:07:26.153142 14963 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:26.153259 14963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:26.153561 14963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:26.156423 14963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "22adf6ed7976489a907121a8986dea13"
format_stamp: "Formatted at 2026-08-12 07:07:26 on dist-test-slave-lgr5"
server_key: "42aeb7fd28f97d2cf926edcba17c514a"
server_key_iv: "2370a5eaa4286d8a4722c010333f8140"
server_key_version: "encryptionkey@0"
I20260812 07:07:26.157147 14963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "22adf6ed7976489a907121a8986dea13"
format_stamp: "Formatted at 2026-08-12 07:07:26 on dist-test-slave-lgr5"
server_key: "42aeb7fd28f97d2cf926edcba17c514a"
server_key_iv: "2370a5eaa4286d8a4722c010333f8140"
server_key_version: "encryptionkey@0"
I20260812 07:07:26.162045 14963 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:07:26.165285 14984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:26.166464 14963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:07:26.166606 14963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "22adf6ed7976489a907121a8986dea13"
format_stamp: "Formatted at 2026-08-12 07:07:26 on dist-test-slave-lgr5"
server_key: "42aeb7fd28f97d2cf926edcba17c514a"
server_key_iv: "2370a5eaa4286d8a4722c010333f8140"
server_key_version: "encryptionkey@0"
I20260812 07:07:26.166783 14963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:26.179402 14963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:26.182940 14963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:26.183208 14963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:26.184065 14963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:26.185360 14963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:26.185442 14963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:26.185557 14963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:26.185613 14963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:26.215152 14963 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:43611
I20260812 07:07:26.215195 15199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:43611 every 8 connection(s)
I20260812 07:07:26.216355 14963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:07:26.217478 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 14963
I20260812 07:07:26.217656 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:07:26.218210 12035 external_mini_cluster.cc:1428] Setting key 68849dd702d35706d30cc7e18b567b60
Aug 12 07:07:26 dist-test-slave-lgr5 krb5kdc[14536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518446, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.194@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:07:26.235929 14956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:26.219026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:44053 (local address 127.11.192.254:36623)
0812 07:07:26.219216 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:26.219222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:26.220448 (+ 1226us) server_negotiation.cc:407] Connection header received
0812 07:07:26.222199 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:26.222205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:26.222283 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:26.222387 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:26.223796 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.224722 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.226191 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.226748 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.226871 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:26.230001 (+ 3130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:26.230007 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:26.230011 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:26.230046 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:26.232943 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:26.233714 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:26.233718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:26.233720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:26.233860 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:26.234325 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:26.234328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:26.234332 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:26.234584 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:26.234703 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:26.235515 (+ 812us) server_negotiation.cc:299] Negotiation successful
0812 07:07:26.235688 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:07:26.237193 15203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:26.219493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:36623 (local address 127.11.192.194:44053)
0812 07:07:26.220289 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:26.220334 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:26.221931 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:26.222573 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:26.222596 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:26.223098 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:26.223631 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:26.223649 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.224856 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.224860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:26.226053 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:26.226066 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.226923 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.226928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:26.227133 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:26.227729 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:26.227751 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:26.229803 (+ 2052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:26.233069 (+ 3266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:26.233075 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:26.233085 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:26.233591 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:26.233962 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:26.233967 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:26.233970 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:26.234214 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:26.234704 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:26.234711 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:26.235105 (+ 394us) client_negotiation.cc:769] Sending connection context
0812 07:07:26.235377 (+ 272us) client_negotiation.cc:240] Negotiation successful
0812 07:07:26.235850 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":302,"threads_started":1}
I20260812 07:07:26.238623 15200 heartbeater.cc:347] Connected to a master server at 127.11.192.254:36623
I20260812 07:07:26.238895 15200 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:26.239610 15200 heartbeater.cc:511] Master 127.11.192.254:36623 requested a full tablet report, sending...
I20260812 07:07:26.241025 14606 ts_manager.cc:194] Registered new tserver with Master: 22adf6ed7976489a907121a8986dea13 (127.11.192.194:43611)
I20260812 07:07:26.241901 14606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.194@KRBTEST.COM'} at 127.11.192.194:44053
WARNING: no policy specified for kudu/127.11.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
Entry for principal kudu/127.11.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab.
I20260812 07:07:26.282318 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36623
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:34207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:26.597223 15207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:26.597479 15207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:26.597611 15207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:26.604504 15207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:26.604609 15207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:26.604754 15207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:07:26.613938 15207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:34207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.195.keytab
--principal=kudu/127.11.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:36623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:26.615091 15207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:26.616387 15207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:26.628176 15217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:26.628249 15221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:26.628273 15218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:26.629230 15207 server_base.cc:1034] running on GCE node
I20260812 07:07:26.629765 15207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:26.630604 15207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:26.631790 15207 hybrid_clock.cc:648] HybridClock initialized: now 1786518446631748 us; error 36 us; skew 500 ppm
I20260812 07:07:26.632093 15207 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:26 dist-test-slave-lgr5 krb5kdc[14536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518446, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:26.634977 15207 init.cc:377] Logged in from keytab as kudu/127.11.192.195@KRBTEST.COM (short username kudu)
I20260812 07:07:26.636245 15207 webserver.cc:460] Webserver started at http://127.11.192.195:32875/ using document root <none> and password file <none>
I20260812 07:07:26.636822 15207 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:26.636919 15207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:26.637171 15207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:26.639806 15207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "aa48e990a4fe4097b3bd137b7395e3ff"
format_stamp: "Formatted at 2026-08-12 07:07:26 on dist-test-slave-lgr5"
server_key: "c621ef8023431a552e339ebcf6cf9554"
server_key_iv: "e1758e850a06d22627482dfa307e0f9e"
server_key_version: "encryptionkey@0"
I20260812 07:07:26.640548 15207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "aa48e990a4fe4097b3bd137b7395e3ff"
format_stamp: "Formatted at 2026-08-12 07:07:26 on dist-test-slave-lgr5"
server_key: "c621ef8023431a552e339ebcf6cf9554"
server_key_iv: "e1758e850a06d22627482dfa307e0f9e"
server_key_version: "encryptionkey@0"
I20260812 07:07:26.645152 15207 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:07:26.648278 15234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:26.649446 15207 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 07:07:26.649567 15207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "aa48e990a4fe4097b3bd137b7395e3ff"
format_stamp: "Formatted at 2026-08-12 07:07:26 on dist-test-slave-lgr5"
server_key: "c621ef8023431a552e339ebcf6cf9554"
server_key_iv: "e1758e850a06d22627482dfa307e0f9e"
server_key_version: "encryptionkey@0"
I20260812 07:07:26.649719 15207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:26.663600 15207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:26.667343 15207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:26.667570 15207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:26.668597 15207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:26.669832 15207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:26.669905 15207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:26.669991 15207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:26.670033 15207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:26.698793 15207 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:38563
I20260812 07:07:26.698833 15417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:38563 every 8 connection(s)
I20260812 07:07:26.699968 15207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:07:26.705958 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 15207
I20260812 07:07:26.706146 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:07:26.706638 12035 external_mini_cluster.cc:1428] Setting key ec0bc5aa0969307f0419b496dce5bf7e
Aug 12 07:07:26 dist-test-slave-lgr5 krb5kdc[14536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518446, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.195@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:07:26.719008 14956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:26.702287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:45119 (local address 127.11.192.254:36623)
0812 07:07:26.702486 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:26.702491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:26.703683 (+ 1192us) server_negotiation.cc:407] Connection header received
0812 07:07:26.706054 (+ 2371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:26.706060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:26.706179 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:26.706284 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:26.707930 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.708835 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.710361 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.710749 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.710858 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:26.714023 (+ 3165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:26.714028 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:26.714031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:26.714061 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:26.716334 (+ 2273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:26.716954 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:26.716959 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:26.716961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:26.717075 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:26.717463 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:26.717467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:26.717471 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:26.717718 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:26.717846 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:26.718460 (+ 614us) server_negotiation.cc:299] Negotiation successful
0812 07:07:26.718668 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 07:07:26.719866 15423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:26.702756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:36623 (local address 127.11.192.195:45119)
0812 07:07:26.703523 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:26.703568 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:26.705702 (+ 2134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:26.706589 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:26.706596 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:26.707119 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:26.707737 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:26.707763 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.708987 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.708993 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:26.710224 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:26.710237 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.710960 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.710968 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:26.711201 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:26.711782 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:07:26.711803 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:26.713823 (+ 2020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:26.716460 (+ 2637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:26.716468 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:26.716482 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:26.716843 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:26.717176 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:07:26.717181 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:07:26.717185 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:07:26.717352 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:07:26.717838 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:26.717845 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:26.718125 (+ 280us) client_negotiation.cc:769] Sending connection context
0812 07:07:26.718339 (+ 214us) client_negotiation.cc:240] Negotiation successful
0812 07:07:26.718791 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":251,"threads_started":1}
I20260812 07:07:26.721123 15420 heartbeater.cc:347] Connected to a master server at 127.11.192.254:36623
I20260812 07:07:26.721388 15420 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:26.722025 15420 heartbeater.cc:511] Master 127.11.192.254:36623 requested a full tablet report, sending...
I20260812 07:07:26.723232 14606 ts_manager.cc:194] Registered new tserver with Master: aa48e990a4fe4097b3bd137b7395e3ff (127.11.192.195:38563)
I20260812 07:07:26.723949 14606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.192.195@KRBTEST.COM'} at 127.11.192.195:45119
I20260812 07:07:26.733178 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:07:26.748153 14956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:26.737149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46368 (local address 127.11.192.254:36623)
0812 07:07:26.737297 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:26.737301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:26.737733 (+ 432us) server_negotiation.cc:407] Connection header received
0812 07:07:26.737859 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:26.737863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:26.737919 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:26.737989 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:26.739378 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.740053 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.741261 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.741671 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.741758 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:26.743778 (+ 2020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:26.743782 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:26.743785 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:26.743813 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:26.746161 (+ 2348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:26.746676 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:26.746679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:26.746682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:26.746799 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:26.747070 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:26.747072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:26.747074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:26.747356 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:26.747528 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:26.747758 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 07:07:26.747919 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:07:26.751490 14606 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46368:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:07:26.758586 14606 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:07:26.762959 14952 heartbeater.cc:503] Master 127.11.192.254:36623 was elected leader, sending a full tablet report...
I20260812 07:07:26.781250 15445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:26.771917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:38563 (local address 127.0.0.1:44874)
0812 07:07:26.772813 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:26.772838 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:26.772923 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:26.774049 (+ 1126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:26.774053 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:26.774094 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:26.774643 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:26.774657 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.776194 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.776199 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:26.778851 (+ 2652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:26.778862 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.780228 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.780235 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:26.780422 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:26.780473 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 07:07:26.780604 (+ 131us) client_negotiation.cc:240] Negotiation successful
0812 07:07:26.780779 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"mutex_wait_us":87,"spinlock_wait_cycles":5888,"thread_start_us":500,"threads_started":1}
I20260812 07:07:26.781277 15442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:26.770376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54712 (local address 127.11.192.194:43611)
0812 07:07:26.772370 (+ 1994us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:26.772378 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:26.772771 (+ 393us) server_negotiation.cc:407] Connection header received
0812 07:07:26.773178 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:26.773187 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:26.773421 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:26.773596 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:26.774793 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.776374 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.778827 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.780415 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.780572 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:26.780746 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:26.780932 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 07:07:26.781053 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1832,"thread_start_us":2093,"threads_started":1}
I20260812 07:07:26.781399 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:26.769896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:43611 (local address 127.0.0.1:54712)
0812 07:07:26.772614 (+ 2718us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:26.772665 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:26.772922 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:26.774042 (+ 1120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:26.774046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:26.774085 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:26.774588 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:26.774606 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.776495 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.776499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:26.778697 (+ 2198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:26.778711 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.780605 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.780611 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:26.780763 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:26.780792 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 07:07:26.780880 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 07:07:26.780999 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2473,"mutex_wait_us":49,"spinlock_wait_cycles":768,"thread_start_us":2752,"threads_started":1}
I20260812 07:07:26.782471 15441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:26.769942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37472 (local address 127.11.192.193:40809)
0812 07:07:26.771355 (+ 1413us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:26.771366 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:26.772959 (+ 1593us) server_negotiation.cc:407] Connection header received
0812 07:07:26.773234 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:26.773241 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:26.773476 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:26.773592 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:26.775025 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.776534 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.778786 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.780401 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.780604 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:26.780753 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:26.782011 (+ 1258us) server_negotiation.cc:299] Negotiation successful
0812 07:07:26.782160 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1225,"thread_start_us":1055,"threads_started":1}
I20260812 07:07:26.783190 15438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:26.769765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:40809 (local address 127.0.0.1:37472)
0812 07:07:26.772837 (+ 3072us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:26.772863 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:26.773065 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:26.773840 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:26.773847 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:26.773879 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:26.774887 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:26.774896 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.776660 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.776666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:26.778641 (+ 1975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:26.778654 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.781650 (+ 2996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.781656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:26.781818 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:26.781847 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 07:07:26.782722 (+ 875us) client_negotiation.cc:240] Negotiation successful
0812 07:07:26.782857 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2226,"mutex_wait_us":28,"thread_start_us":2000,"threads_started":1}
I20260812 07:07:26.784423 15447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:26.772183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44874 (local address 127.11.192.195:38563)
0812 07:07:26.773522 (+ 1339us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:26.773530 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:26.773559 (+ 29us) server_negotiation.cc:407] Connection header received
0812 07:07:26.773645 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:26.773652 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:26.773840 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:26.774173 (+ 333us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:26.775031 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.776048 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.778977 (+ 2929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.780015 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.780230 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:26.780430 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:26.781868 (+ 1438us) server_negotiation.cc:299] Negotiation successful
0812 07:07:26.782037 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1182,"thread_start_us":1019,"threads_started":1}
I20260812 07:07:26.785283 15074 tablet_service.cc:1467] Processing CreateTablet for tablet 4a4f40445e904736963babbdc61cccfc (DEFAULT_TABLE table=test-table [id=99b7810c4c1b42dda2314c3ed9dbd5b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:26.786442 15074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a4f40445e904736963babbdc61cccfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:26.786319 14835 tablet_service.cc:1467] Processing CreateTablet for tablet 4a4f40445e904736963babbdc61cccfc (DEFAULT_TABLE table=test-table [id=99b7810c4c1b42dda2314c3ed9dbd5b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:26.787431 14835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a4f40445e904736963babbdc61cccfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:26.794772 15317 tablet_service.cc:1467] Processing CreateTablet for tablet 4a4f40445e904736963babbdc61cccfc (DEFAULT_TABLE table=test-table [id=99b7810c4c1b42dda2314c3ed9dbd5b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:26.795815 15317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a4f40445e904736963babbdc61cccfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:26.799651 15450 tablet_bootstrap.cc:492] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7: Bootstrap starting.
I20260812 07:07:26.800424 15452 tablet_bootstrap.cc:492] T 4a4f40445e904736963babbdc61cccfc P 22adf6ed7976489a907121a8986dea13: Bootstrap starting.
I20260812 07:07:26.803505 15452 tablet_bootstrap.cc:654] T 4a4f40445e904736963babbdc61cccfc P 22adf6ed7976489a907121a8986dea13: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:26.803819 15450 tablet_bootstrap.cc:654] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:26.804538 15452 log.cc:824] T 4a4f40445e904736963babbdc61cccfc P 22adf6ed7976489a907121a8986dea13: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:26.805292 15450 log.cc:824] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:26.807294 15452 tablet_bootstrap.cc:492] T 4a4f40445e904736963babbdc61cccfc P 22adf6ed7976489a907121a8986dea13: No bootstrap required, opened a new log
I20260812 07:07:26.807528 15452 ts_tablet_manager.cc:1397] T 4a4f40445e904736963babbdc61cccfc P 22adf6ed7976489a907121a8986dea13: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 07:07:26.808426 15455 tablet_bootstrap.cc:492] T 4a4f40445e904736963babbdc61cccfc P aa48e990a4fe4097b3bd137b7395e3ff: Bootstrap starting.
I20260812 07:07:26.808706 15450 tablet_bootstrap.cc:492] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7: No bootstrap required, opened a new log
I20260812 07:07:26.808979 15450 ts_tablet_manager.cc:1397] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.005s
I20260812 07:07:26.811794 15455 tablet_bootstrap.cc:654] T 4a4f40445e904736963babbdc61cccfc P aa48e990a4fe4097b3bd137b7395e3ff: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:26.812928 15455 log.cc:824] T 4a4f40445e904736963babbdc61cccfc P aa48e990a4fe4097b3bd137b7395e3ff: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:26.815728 15455 tablet_bootstrap.cc:492] T 4a4f40445e904736963babbdc61cccfc P aa48e990a4fe4097b3bd137b7395e3ff: No bootstrap required, opened a new log
I20260812 07:07:26.815987 15455 ts_tablet_manager.cc:1397] T 4a4f40445e904736963babbdc61cccfc P aa48e990a4fe4097b3bd137b7395e3ff: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 07:07:26.816506 15452 raft_consensus.cc:348] T 4a4f40445e904736963babbdc61cccfc P 22adf6ed7976489a907121a8986dea13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48ba28d907bd414ebccdd5f39e6aefd7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40809 } } peers { permanent_uuid: "22adf6ed7976489a907121a8986dea13" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 43611 } } peers { permanent_uuid: "aa48e990a4fe4097b3bd137b7395e3ff" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 38563 } }
I20260812 07:07:26.816833 15452 raft_consensus.cc:374] T 4a4f40445e904736963babbdc61cccfc P 22adf6ed7976489a907121a8986dea13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:26.816943 15452 raft_consensus.cc:729] T 4a4f40445e904736963babbdc61cccfc P 22adf6ed7976489a907121a8986dea13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22adf6ed7976489a907121a8986dea13, State: Initialized, Role: FOLLOWER
I20260812 07:07:26.817034 15450 raft_consensus.cc:348] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48ba28d907bd414ebccdd5f39e6aefd7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40809 } } peers { permanent_uuid: "22adf6ed7976489a907121a8986dea13" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 43611 } } peers { permanent_uuid: "aa48e990a4fe4097b3bd137b7395e3ff" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 38563 } }
I20260812 07:07:26.817446 15450 raft_consensus.cc:374] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:26.817543 15450 raft_consensus.cc:729] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48ba28d907bd414ebccdd5f39e6aefd7, State: Initialized, Role: FOLLOWER
I20260812 07:07:26.817494 15452 consensus_queue.cc:260] T 4a4f40445e904736963babbdc61cccfc P 22adf6ed7976489a907121a8986dea13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48ba28d907bd414ebccdd5f39e6aefd7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40809 } } peers { permanent_uuid: "22adf6ed7976489a907121a8986dea13" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 43611 } } peers { permanent_uuid: "aa48e990a4fe4097b3bd137b7395e3ff" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 38563 } }
I20260812 07:07:26.818082 15450 consensus_queue.cc:260] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48ba28d907bd414ebccdd5f39e6aefd7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40809 } } peers { permanent_uuid: "22adf6ed7976489a907121a8986dea13" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 43611 } } peers { permanent_uuid: "aa48e990a4fe4097b3bd137b7395e3ff" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 38563 } }
I20260812 07:07:26.818753 15200 heartbeater.cc:503] Master 127.11.192.254:36623 was elected leader, sending a full tablet report...
I20260812 07:07:26.819145 15452 ts_tablet_manager.cc:1428] T 4a4f40445e904736963babbdc61cccfc P 22adf6ed7976489a907121a8986dea13: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260812 07:07:26.821470 15450 ts_tablet_manager.cc:1428] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260812 07:07:26.826000 15455 raft_consensus.cc:348] T 4a4f40445e904736963babbdc61cccfc P aa48e990a4fe4097b3bd137b7395e3ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48ba28d907bd414ebccdd5f39e6aefd7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40809 } } peers { permanent_uuid: "22adf6ed7976489a907121a8986dea13" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 43611 } } peers { permanent_uuid: "aa48e990a4fe4097b3bd137b7395e3ff" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 38563 } }
I20260812 07:07:26.826320 15455 raft_consensus.cc:374] T 4a4f40445e904736963babbdc61cccfc P aa48e990a4fe4097b3bd137b7395e3ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:26.826417 15455 raft_consensus.cc:729] T 4a4f40445e904736963babbdc61cccfc P aa48e990a4fe4097b3bd137b7395e3ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa48e990a4fe4097b3bd137b7395e3ff, State: Initialized, Role: FOLLOWER
I20260812 07:07:26.826864 15455 consensus_queue.cc:260] T 4a4f40445e904736963babbdc61cccfc P aa48e990a4fe4097b3bd137b7395e3ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48ba28d907bd414ebccdd5f39e6aefd7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40809 } } peers { permanent_uuid: "22adf6ed7976489a907121a8986dea13" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 43611 } } peers { permanent_uuid: "aa48e990a4fe4097b3bd137b7395e3ff" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 38563 } }
I20260812 07:07:26.827772 15420 heartbeater.cc:503] Master 127.11.192.254:36623 was elected leader, sending a full tablet report...
I20260812 07:07:26.828302 15455 ts_tablet_manager.cc:1428] T 4a4f40445e904736963babbdc61cccfc P aa48e990a4fe4097b3bd137b7395e3ff: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
W20260812 07:07:26.951538 15422 tablet.cc:2369] T 4a4f40445e904736963babbdc61cccfc P aa48e990a4fe4097b3bd137b7395e3ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:07:26.968849 15202 tablet.cc:2369] T 4a4f40445e904736963babbdc61cccfc P 22adf6ed7976489a907121a8986dea13: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:07:26.972362 15459 raft_consensus.cc:482] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:07:26.972666 15459 raft_consensus.cc:504] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48ba28d907bd414ebccdd5f39e6aefd7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40809 } } peers { permanent_uuid: "22adf6ed7976489a907121a8986dea13" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 43611 } } peers { permanent_uuid: "aa48e990a4fe4097b3bd137b7395e3ff" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 38563 } }
I20260812 07:07:26.974375 15459 leader_election.cc:288] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22adf6ed7976489a907121a8986dea13 (127.11.192.194:43611), aa48e990a4fe4097b3bd137b7395e3ff (127.11.192.195:38563)
I20260812 07:07:26.981611 15442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:26.974757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:52737 (local address 127.11.192.194:43611)
0812 07:07:26.974999 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:26.975006 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:26.976038 (+ 1032us) server_negotiation.cc:407] Connection header received
0812 07:07:26.976263 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:26.976268 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:26.976334 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:26.976399 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:26.977054 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.977787 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.979837 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.980637 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.980763 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:26.980837 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:26.981228 (+ 391us) server_negotiation.cc:299] Negotiation successful
0812 07:07:26.981353 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 07:07:26.981624 15447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:26.974819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:45005 (local address 127.11.192.195:38563)
0812 07:07:26.975025 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:26.975031 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:26.976110 (+ 1079us) server_negotiation.cc:407] Connection header received
0812 07:07:26.976275 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:26.976279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:26.976346 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:26.976440 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:26.977105 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.977871 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.979766 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.980649 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.980829 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:26.980911 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:26.981233 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 07:07:26.981360 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:07:26.981761 15465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:26.975031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:38563 (local address 127.11.192.193:45005)
0812 07:07:26.975969 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:26.975991 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:26.976143 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:26.976468 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:26.976472 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:26.976496 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:26.976934 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:26.976944 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.978030 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.978034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:26.979600 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:26.979616 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.980910 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.980915 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:26.981047 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:26.981106 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 07:07:26.981187 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 07:07:26.981319 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":831,"mutex_wait_us":67,"thread_start_us":438,"threads_started":1}
I20260812 07:07:26.981853 15464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:26.974989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:43611 (local address 127.11.192.193:52737)
0812 07:07:26.975921 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:26.975944 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:26.976108 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:26.976447 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:26.976452 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:26.976477 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:26.976904 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:26.976919 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.977924 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.977930 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:26.979672 (+ 1742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:26.979690 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:26.980876 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:26.980883 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:26.981046 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:26.981093 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 07:07:26.981185 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 07:07:26.981340 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"mutex_wait_us":92,"thread_start_us":526,"threads_started":1}
I20260812 07:07:26.982874 15116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a4f40445e904736963babbdc61cccfc" candidate_uuid: "48ba28d907bd414ebccdd5f39e6aefd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22adf6ed7976489a907121a8986dea13" is_pre_election: true
I20260812 07:07:26.982874 15352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a4f40445e904736963babbdc61cccfc" candidate_uuid: "48ba28d907bd414ebccdd5f39e6aefd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa48e990a4fe4097b3bd137b7395e3ff" is_pre_election: true
I20260812 07:07:26.983459 15116 raft_consensus.cc:2454] T 4a4f40445e904736963babbdc61cccfc P 22adf6ed7976489a907121a8986dea13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48ba28d907bd414ebccdd5f39e6aefd7 in term 0.
I20260812 07:07:26.983458 15352 raft_consensus.cc:2454] T 4a4f40445e904736963babbdc61cccfc P aa48e990a4fe4097b3bd137b7395e3ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48ba28d907bd414ebccdd5f39e6aefd7 in term 0.
I20260812 07:07:26.984196 14752 leader_election.cc:302] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48ba28d907bd414ebccdd5f39e6aefd7, aa48e990a4fe4097b3bd137b7395e3ff; no voters:
I20260812 07:07:26.984581 15459 raft_consensus.cc:2781] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:07:26.984735 15459 raft_consensus.cc:482] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:07:26.984845 15459 raft_consensus.cc:3037] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7 [term 0 FOLLOWER]: Advancing to term 1
W20260812 07:07:26.986145 14955 tablet.cc:2369] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:07:26.987003 15459 raft_consensus.cc:504] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48ba28d907bd414ebccdd5f39e6aefd7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40809 } } peers { permanent_uuid: "22adf6ed7976489a907121a8986dea13" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 43611 } } peers { permanent_uuid: "aa48e990a4fe4097b3bd137b7395e3ff" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 38563 } }
I20260812 07:07:26.987550 15459 leader_election.cc:288] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7 [CANDIDATE]: Term 1 election: Requested vote from peers 22adf6ed7976489a907121a8986dea13 (127.11.192.194:43611), aa48e990a4fe4097b3bd137b7395e3ff (127.11.192.195:38563)
I20260812 07:07:26.987977 15116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a4f40445e904736963babbdc61cccfc" candidate_uuid: "48ba28d907bd414ebccdd5f39e6aefd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22adf6ed7976489a907121a8986dea13"
I20260812 07:07:26.988003 15352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a4f40445e904736963babbdc61cccfc" candidate_uuid: "48ba28d907bd414ebccdd5f39e6aefd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa48e990a4fe4097b3bd137b7395e3ff"
I20260812 07:07:26.988205 15116 raft_consensus.cc:3037] T 4a4f40445e904736963babbdc61cccfc P 22adf6ed7976489a907121a8986dea13 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:26.988205 15352 raft_consensus.cc:3037] T 4a4f40445e904736963babbdc61cccfc P aa48e990a4fe4097b3bd137b7395e3ff [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:26.991279 15116 raft_consensus.cc:2454] T 4a4f40445e904736963babbdc61cccfc P 22adf6ed7976489a907121a8986dea13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48ba28d907bd414ebccdd5f39e6aefd7 in term 1.
I20260812 07:07:26.991279 15352 raft_consensus.cc:2454] T 4a4f40445e904736963babbdc61cccfc P aa48e990a4fe4097b3bd137b7395e3ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48ba28d907bd414ebccdd5f39e6aefd7 in term 1.
I20260812 07:07:26.991698 14755 leader_election.cc:302] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 22adf6ed7976489a907121a8986dea13, 48ba28d907bd414ebccdd5f39e6aefd7, aa48e990a4fe4097b3bd137b7395e3ff; no voters:
I20260812 07:07:26.991994 15459 raft_consensus.cc:2781] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:26.992452 15459 raft_consensus.cc:686] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7 [term 1 LEADER]: Becoming Leader. State: Replica: 48ba28d907bd414ebccdd5f39e6aefd7, State: Running, Role: LEADER
I20260812 07:07:26.992975 15459 consensus_queue.cc:237] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48ba28d907bd414ebccdd5f39e6aefd7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40809 } } peers { permanent_uuid: "22adf6ed7976489a907121a8986dea13" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 43611 } } peers { permanent_uuid: "aa48e990a4fe4097b3bd137b7395e3ff" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 38563 } }
I20260812 07:07:26.997310 14604 catalog_manager.cc:5495] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48ba28d907bd414ebccdd5f39e6aefd7 (127.11.192.193). New cstate: current_term: 1 leader_uuid: "48ba28d907bd414ebccdd5f39e6aefd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48ba28d907bd414ebccdd5f39e6aefd7" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 40809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "22adf6ed7976489a907121a8986dea13" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa48e990a4fe4097b3bd137b7395e3ff" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 38563 } health_report { overall_health: UNKNOWN } } }
I20260812 07:07:27.019845 15441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:27.014906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37482 (local address 127.11.192.193:40809)
0812 07:07:27.015089 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:27.015093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:27.015109 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:07:27.015236 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:27.015240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:27.015345 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:27.015467 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:07:27.015772 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:27.016553 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:27.017882 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:27.018280 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:27.018382 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:27.018831 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:07:27.019400 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:07:27.019502 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:27.019586 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 07:07:27.019648 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 07:07:27.027663 15116 raft_consensus.cc:1261] T 4a4f40445e904736963babbdc61cccfc P 22adf6ed7976489a907121a8986dea13 [term 1 FOLLOWER]: Refusing update from remote peer 48ba28d907bd414ebccdd5f39e6aefd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:07:27.027667 15352 raft_consensus.cc:1261] T 4a4f40445e904736963babbdc61cccfc P aa48e990a4fe4097b3bd137b7395e3ff [term 1 FOLLOWER]: Refusing update from remote peer 48ba28d907bd414ebccdd5f39e6aefd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:07:27.028751 15459 consensus_queue.cc:1035] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa48e990a4fe4097b3bd137b7395e3ff" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 38563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:07:27.029059 15466 consensus_queue.cc:1035] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22adf6ed7976489a907121a8986dea13" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:07:27.040493 15475 mvcc.cc:205] Tried to move back new op lower bound from 7317579559014932480 to 7317579558886789120. Current Snapshot: MvccSnapshot[applied={T|T < 7317579559014932480}]
I20260812 07:07:27.042804 15476 mvcc.cc:205] Tried to move back new op lower bound from 7317579559014932480 to 7317579558886789120. Current Snapshot: MvccSnapshot[applied={T|T < 7317579559014932480}]
I20260812 07:07:27.052145 14606 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:07:27.052402 14606 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:07:27.056010 14606 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b518c8d0f98448d4b87f08807a3f3f0a: Sending DeleteTablet for 3 replicas of tablet 4a4f40445e904736963babbdc61cccfc
I20260812 07:07:27.057068 15074 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a4f40445e904736963babbdc61cccfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:27 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:54712
I20260812 07:07:27.057132 14835 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a4f40445e904736963babbdc61cccfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:27 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:37472
I20260812 07:07:27.057714 15317 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a4f40445e904736963babbdc61cccfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:27 UTC) from {username='kudu', principal='kudu/127.11.192.254@KRBTEST.COM'} at 127.0.0.1:44874
I20260812 07:07:27.058002 15486 tablet_replica.cc:330] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7: stopping tablet replica
I20260812 07:07:27.058095 15487 tablet_replica.cc:330] T 4a4f40445e904736963babbdc61cccfc P 22adf6ed7976489a907121a8986dea13: stopping tablet replica
I20260812 07:07:27.058452 15487 raft_consensus.cc:2229] T 4a4f40445e904736963babbdc61cccfc P 22adf6ed7976489a907121a8986dea13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:07:27.058470 15486 raft_consensus.cc:2229] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:07:27.058900 15487 raft_consensus.cc:2258] T 4a4f40445e904736963babbdc61cccfc P 22adf6ed7976489a907121a8986dea13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:27.059265 15486 raft_consensus.cc:2258] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:27.059365 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 14722
I20260812 07:07:27.060979 15487 ts_tablet_manager.cc:1905] T 4a4f40445e904736963babbdc61cccfc P 22adf6ed7976489a907121a8986dea13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:27.061517 15486 ts_tablet_manager.cc:1905] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:27.067610 15487 ts_tablet_manager.cc:1918] T 4a4f40445e904736963babbdc61cccfc P 22adf6ed7976489a907121a8986dea13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:07:27.067765 15487 log.cc:1196] T 4a4f40445e904736963babbdc61cccfc P 22adf6ed7976489a907121a8986dea13: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/4a4f40445e904736963babbdc61cccfc
I20260812 07:07:27.068320 15487 ts_tablet_manager.cc:1939] T 4a4f40445e904736963babbdc61cccfc P 22adf6ed7976489a907121a8986dea13: Deleting consensus metadata
I20260812 07:07:27.068771 15486 ts_tablet_manager.cc:1918] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:07:27.068934 15486 log.cc:1196] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-0/wal/wals/4a4f40445e904736963babbdc61cccfc
I20260812 07:07:27.069453 15486 ts_tablet_manager.cc:1939] T 4a4f40445e904736963babbdc61cccfc P 48ba28d907bd414ebccdd5f39e6aefd7: Deleting consensus metadata
I20260812 07:07:27.070000 14584 catalog_manager.cc:4841] TS 22adf6ed7976489a907121a8986dea13 (127.11.192.194:43611): tablet 4a4f40445e904736963babbdc61cccfc (table test-table [id=99b7810c4c1b42dda2314c3ed9dbd5b2]) successfully deleted
I20260812 07:07:27.070703 15489 tablet_replica.cc:330] T 4a4f40445e904736963babbdc61cccfc P aa48e990a4fe4097b3bd137b7395e3ff: stopping tablet replica
I20260812 07:07:27.071003 14585 catalog_manager.cc:4841] TS 48ba28d907bd414ebccdd5f39e6aefd7 (127.11.192.193:40809): tablet 4a4f40445e904736963babbdc61cccfc (table test-table [id=99b7810c4c1b42dda2314c3ed9dbd5b2]) successfully deleted
I20260812 07:07:27.071162 15489 raft_consensus.cc:2229] T 4a4f40445e904736963babbdc61cccfc P aa48e990a4fe4097b3bd137b7395e3ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:07:27.071650 15489 raft_consensus.cc:2258] T 4a4f40445e904736963babbdc61cccfc P aa48e990a4fe4097b3bd137b7395e3ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:27.073401 15489 ts_tablet_manager.cc:1905] T 4a4f40445e904736963babbdc61cccfc P aa48e990a4fe4097b3bd137b7395e3ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:27.080008 15489 ts_tablet_manager.cc:1918] T 4a4f40445e904736963babbdc61cccfc P aa48e990a4fe4097b3bd137b7395e3ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:07:27.080211 15489 log.cc:1196] T 4a4f40445e904736963babbdc61cccfc P aa48e990a4fe4097b3bd137b7395e3ff: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/4a4f40445e904736963babbdc61cccfc
I20260812 07:07:27.080725 15489 ts_tablet_manager.cc:1939] T 4a4f40445e904736963babbdc61cccfc P aa48e990a4fe4097b3bd137b7395e3ff: Deleting consensus metadata
Aug 12 07:07:27 dist-test-slave-lgr5 krb5kdc[14536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.193@KRBTEST.COM
I20260812 07:07:27.082350 14582 catalog_manager.cc:4841] TS aa48e990a4fe4097b3bd137b7395e3ff (127.11.192.195:38563): tablet 4a4f40445e904736963babbdc61cccfc (table test-table [id=99b7810c4c1b42dda2314c3ed9dbd5b2]) successfully deleted
I20260812 07:07:27.088639 15441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:27.060831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37486 (local address 127.11.192.193:40809)
0812 07:07:27.060993 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:27.060999 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:27.074595 (+ 13596us) server_negotiation.cc:407] Connection header received
0812 07:07:27.074760 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:27.074765 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:27.074858 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:27.074965 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:27.076236 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:27.077047 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:27.078287 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:27.078748 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:27.078857 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:27.083009 (+ 4152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:27.083023 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:27.083039 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:27.083086 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:27.085984 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:27.086625 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:27.086632 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:27.086634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:27.086776 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:27.087139 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:27.087143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:27.087145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:27.087497 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:27.087800 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:27.088122 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 07:07:27.088353 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:07:27.089362 14943 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:27.187829 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 14722
I20260812 07:07:27.215456 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 14963
Aug 12 07:07:27 dist-test-slave-lgr5 krb5kdc[14536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.194@KRBTEST.COM
I20260812 07:07:27.232060 15442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:27.216294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54714 (local address 127.11.192.194:43611)
0812 07:07:27.216552 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:27.216558 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:27.216581 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:07:27.216649 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:27.216657 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:27.216736 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:27.216878 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:27.218051 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:27.219228 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:27.220576 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:27.221212 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:27.221330 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:27.225366 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:27.225379 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:27.225395 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:27.225440 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:27.229146 (+ 3706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:27.229915 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:27.229924 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:27.229927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:27.230085 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:27.230454 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:27.230458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:27.230461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:27.230972 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:27.231285 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:27.231489 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 07:07:27.231701 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 07:07:27.232879 15193 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:27.324797 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 14963
I20260812 07:07:27.357384 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 15207
Aug 12 07:07:27 dist-test-slave-lgr5 krb5kdc[14536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.195@KRBTEST.COM
I20260812 07:07:27.375530 15447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:27.358700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44890 (local address 127.11.192.195:38563)
0812 07:07:27.358903 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:27.358911 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:27.358932 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:07:27.359005 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:27.359011 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:27.359098 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:27.359205 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:27.361310 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:27.362453 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:27.364526 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:27.365128 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:27.365240 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:27.370765 (+ 5525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:27.370782 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:27.370800 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:27.370834 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:27.373345 (+ 2511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:27.373943 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:27.373949 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:27.373951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:27.374093 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:27.374365 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:27.374369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:27.374370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:27.374708 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:27.374940 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:27.375134 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 07:07:27.375290 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:07:27.376260 15410 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:27.462136 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 15207
I20260812 07:07:27.489357 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 14561
I20260812 07:07:27.490406 14665 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:27.598285 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 14561
2026-08-12T07:07:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4409 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T07:07:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:27Z Disabled control of system clock
I20260812 07:07:27.690233 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:43487
--webserver_interface=127.11.192.254
--webserver_port=0
--builtin_ntp_servers=127.11.192.212:43953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:43487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:07:28.016618 15506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:28.016875 15506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:28.017000 15506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:28.023913 15506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:28.024006 15506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:28.024070 15506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:28.024160 15506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:28.024219 15506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:28.032919 15506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:43487
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:43487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:28.034070 15506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:28.035789 15506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:28.046598 15519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:28.046677 15513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:28.046653 15515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:28.047497 15506 server_base.cc:1034] running on GCE node
I20260812 07:07:28.048200 15506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:28.049428 15506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:28.050619 15506 hybrid_clock.cc:648] HybridClock initialized: now 1786518448050582 us; error 37 us; skew 500 ppm
I20260812 07:07:28.050907 15506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:07:28.053011 15506 webserver.cc:460] Webserver started at http://127.11.192.254:34589/ using document root <none> and password file <none>
I20260812 07:07:28.053599 15506 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:28.053681 15506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:28.053936 15506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:28.056658 15506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "bbceae5efa64427fb346727afed6bcf9"
format_stamp: "Formatted at 2026-08-12 07:07:28 on dist-test-slave-lgr5"
server_key: "562f004b4abf01baaba5c5334d9c4a70"
server_key_iv: "5031b47093483728cd5c1c66459206b6"
server_key_version: "encryptionkey@0"
I20260812 07:07:28.057411 15506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "bbceae5efa64427fb346727afed6bcf9"
format_stamp: "Formatted at 2026-08-12 07:07:28 on dist-test-slave-lgr5"
server_key: "562f004b4abf01baaba5c5334d9c4a70"
server_key_iv: "5031b47093483728cd5c1c66459206b6"
server_key_version: "encryptionkey@0"
I20260812 07:07:28.062079 15506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 07:07:28.065189 15528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:28.066367 15506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:07:28.066509 15506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "bbceae5efa64427fb346727afed6bcf9"
format_stamp: "Formatted at 2026-08-12 07:07:28 on dist-test-slave-lgr5"
server_key: "562f004b4abf01baaba5c5334d9c4a70"
server_key_iv: "5031b47093483728cd5c1c66459206b6"
server_key_version: "encryptionkey@0"
I20260812 07:07:28.066670 15506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:28.087725 15506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:28.088474 15506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:28.088696 15506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:28.105298 15506 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:43487
I20260812 07:07:28.105316 15610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:43487 every 8 connection(s)
I20260812 07:07:28.106468 15506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:28.109897 15611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:28.112891 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 15506
I20260812 07:07:28.113047 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:07:28.113453 12035 external_mini_cluster.cc:1428] Setting key 7c052a6160952b90818fef1967b6605a
I20260812 07:07:28.122685 15611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9: Bootstrap starting.
I20260812 07:07:28.126384 15611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:28.127610 15611 log.cc:824] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:28.127983 15619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:28.115222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48076 (local address 127.11.192.254:43487)
0812 07:07:28.116280 (+ 1058us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:28.116294 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:28.116344 (+ 50us) server_negotiation.cc:407] Connection header received
0812 07:07:28.118336 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:28.118364 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:28.118961 (+ 597us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:28.119376 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:28.119909 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:28.121873 (+ 1964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:28.123156 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:28.124178 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:28.124384 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:28.125369 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:28.125379 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:28.125396 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:28.126055 (+ 659us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:28.126178 (+ 123us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:28.126205 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:28.126330 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:28.126456 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 07:07:28.126803 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":325,"threads_started":1}
I20260812 07:07:28.130445 15611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9: No bootstrap required, opened a new log
I20260812 07:07:28.137557 15611 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbceae5efa64427fb346727afed6bcf9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43487 } }
I20260812 07:07:28.137852 15611 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:28.137923 15611 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbceae5efa64427fb346727afed6bcf9, State: Initialized, Role: FOLLOWER
I20260812 07:07:28.138409 15611 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbceae5efa64427fb346727afed6bcf9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43487 } }
I20260812 07:07:28.138631 15611 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:28.138764 15611 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:28.138907 15611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:28.141228 15611 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbceae5efa64427fb346727afed6bcf9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43487 } }
I20260812 07:07:28.141681 15611 leader_election.cc:302] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbceae5efa64427fb346727afed6bcf9; no voters:
I20260812 07:07:28.142194 15611 leader_election.cc:288] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:28.142313 15628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:28.143013 15628 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9 [term 1 LEADER]: Becoming Leader. State: Replica: bbceae5efa64427fb346727afed6bcf9, State: Running, Role: LEADER
I20260812 07:07:28.143469 15628 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbceae5efa64427fb346727afed6bcf9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43487 } }
I20260812 07:07:28.143991 15611 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:28.146184 15629 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbceae5efa64427fb346727afed6bcf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbceae5efa64427fb346727afed6bcf9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43487 } } }
I20260812 07:07:28.146653 15629 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:28.146162 15631 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbceae5efa64427fb346727afed6bcf9. Latest consensus state: current_term: 1 leader_uuid: "bbceae5efa64427fb346727afed6bcf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbceae5efa64427fb346727afed6bcf9" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43487 } } }
I20260812 07:07:28.146811 15631 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:28.148375 15637 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:28.153232 15637 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:28.160657 15637 catalog_manager.cc:1347] Generated new cluster ID: b1b9322476534dfcb189d8e15d8bfb20
I20260812 07:07:28.160755 15637 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:28.177695 15637 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:28.179351 15637 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:28.198238 15637 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9: Generated new TSK 0
I20260812 07:07:28.199200 15637 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:07:28.213385 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.193:0
--local_ip_for_outbound_sockets=127.11.192.193
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43487
--builtin_ntp_servers=127.11.192.212:43953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:07:28.540997 15666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:28.541247 15666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:28.541360 15666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:28.548427 15666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:28.548520 15666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:28.548659 15666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.193
I20260812 07:07:28.557204 15666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.192.193
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.193
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:28.558280 15666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:28.559937 15666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:28.571944 15683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:28.572026 15676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:28.572002 15678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:28.572952 15666 server_base.cc:1034] running on GCE node
I20260812 07:07:28.573520 15666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:28.574354 15666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:28.575552 15666 hybrid_clock.cc:648] HybridClock initialized: now 1786518448575507 us; error 49 us; skew 500 ppm
I20260812 07:07:28.575837 15666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:07:28.580596 15666 webserver.cc:460] Webserver started at http://127.11.192.193:36877/ using document root <none> and password file <none>
I20260812 07:07:28.581234 15666 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:28.581331 15666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:28.581625 15666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:28.584321 15666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-0/data/instance:
uuid: "f8fb4ebb1c4c452bb77af1aeaf2b0d5e"
format_stamp: "Formatted at 2026-08-12 07:07:28 on dist-test-slave-lgr5"
server_key: "9aa5d8cc5452d3b906e4edc9a585fa3d"
server_key_iv: "f3584942b565c096527f081b7141a1e2"
server_key_version: "encryptionkey@0"
I20260812 07:07:28.585071 15666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance:
uuid: "f8fb4ebb1c4c452bb77af1aeaf2b0d5e"
format_stamp: "Formatted at 2026-08-12 07:07:28 on dist-test-slave-lgr5"
server_key: "9aa5d8cc5452d3b906e4edc9a585fa3d"
server_key_iv: "f3584942b565c096527f081b7141a1e2"
server_key_version: "encryptionkey@0"
I20260812 07:07:28.589715 15666 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:07:28.592953 15692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:28.594115 15666 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:07:28.594264 15666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-0/wal
uuid: "f8fb4ebb1c4c452bb77af1aeaf2b0d5e"
format_stamp: "Formatted at 2026-08-12 07:07:28 on dist-test-slave-lgr5"
server_key: "9aa5d8cc5452d3b906e4edc9a585fa3d"
server_key_iv: "f3584942b565c096527f081b7141a1e2"
server_key_version: "encryptionkey@0"
I20260812 07:07:28.594450 15666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:28.616564 15666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:28.617233 15666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:28.617502 15666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:28.618343 15666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:28.619628 15666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:28.619720 15666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:28.619820 15666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:28.619896 15666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:28.649461 15666 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.193:45665
I20260812 07:07:28.649526 15913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.193:45665 every 8 connection(s)
I20260812 07:07:28.650684 15666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-0/data/info.pb
I20260812 07:07:28.652729 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 15666
I20260812 07:07:28.652886 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-0/wal/instance
I20260812 07:07:28.653362 12035 external_mini_cluster.cc:1428] Setting key b08ff2e67e78f9932ccec7e38fafd017
I20260812 07:07:28.655998 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.194:0
--local_ip_for_outbound_sockets=127.11.192.194
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43487
--builtin_ntp_servers=127.11.192.212:43953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:07:28.662772 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:28.653088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:54821 (local address 127.11.192.254:43487)
0812 07:07:28.653546 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:28.653552 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:28.654817 (+ 1265us) server_negotiation.cc:407] Connection header received
0812 07:07:28.656673 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:28.656679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:28.656757 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:28.656870 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:28.657804 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:28.658708 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:28.660187 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:28.660722 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:28.660840 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:28.662009 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:28.662015 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:28.662019 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:28.662040 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:28.662089 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:28.662104 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:28.662186 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:28.662340 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 07:07:28.662435 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":215,"threads_started":1}
I20260812 07:07:28.663547 15923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:28.653696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43487 (local address 127.11.192.193:54821)
0812 07:07:28.654650 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:28.654705 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:28.656427 (+ 1722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:28.657039 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:28.657046 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:28.657131 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:28.657646 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:28.657661 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:28.658830 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:28.658833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:28.660032 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:28.660044 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:28.660914 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:28.660919 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:28.661108 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:28.661713 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:07:28.661715 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:28.661751 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:07:28.661822 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:28.662181 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:28.662187 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:28.662191 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 07:07:28.662310 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 07:07:28.662606 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":290,"threads_started":1}
I20260812 07:07:28.664915 15917 heartbeater.cc:347] Connected to a master server at 127.11.192.254:43487
I20260812 07:07:28.665213 15917 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:28.666013 15917 heartbeater.cc:511] Master 127.11.192.254:43487 requested a full tablet report, sending...
I20260812 07:07:28.667966 15553 ts_manager.cc:194] Registered new tserver with Master: f8fb4ebb1c4c452bb77af1aeaf2b0d5e (127.11.192.193:45665)
I20260812 07:07:28.669651 15553 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.192.193:54821
W20260812 07:07:28.985846 15926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:28.986093 15926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:28.986186 15926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:28.993046 15926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:28.993131 15926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:28.993238 15926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.194
I20260812 07:07:29.001880 15926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.192.194
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.194
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:29.003018 15926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:29.004731 15926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:29.017102 15933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:29.017041 15935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:29.017076 15942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:29.018110 15926 server_base.cc:1034] running on GCE node
I20260812 07:07:29.018657 15926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:29.019586 15926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:29.020782 15926 hybrid_clock.cc:648] HybridClock initialized: now 1786518449020734 us; error 34 us; skew 500 ppm
I20260812 07:07:29.021080 15926 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:07:29.023085 15926 webserver.cc:460] Webserver started at http://127.11.192.194:34601/ using document root <none> and password file <none>
I20260812 07:07:29.023707 15926 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:29.023808 15926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:29.024148 15926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:29.026854 15926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-1/data/instance:
uuid: "c5afefd62fa84c28b9c119728c7abbf8"
format_stamp: "Formatted at 2026-08-12 07:07:29 on dist-test-slave-lgr5"
server_key: "bc101392d6b98b1628ab0200df8fea9a"
server_key_iv: "43fcd648a1d45a2c20efaea7c256f1ef"
server_key_version: "encryptionkey@0"
I20260812 07:07:29.027622 15926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance:
uuid: "c5afefd62fa84c28b9c119728c7abbf8"
format_stamp: "Formatted at 2026-08-12 07:07:29 on dist-test-slave-lgr5"
server_key: "bc101392d6b98b1628ab0200df8fea9a"
server_key_iv: "43fcd648a1d45a2c20efaea7c256f1ef"
server_key_version: "encryptionkey@0"
I20260812 07:07:29.032456 15926 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 07:07:29.035614 15955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:29.036839 15926 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:07:29.037000 15926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-1/wal
uuid: "c5afefd62fa84c28b9c119728c7abbf8"
format_stamp: "Formatted at 2026-08-12 07:07:29 on dist-test-slave-lgr5"
server_key: "bc101392d6b98b1628ab0200df8fea9a"
server_key_iv: "43fcd648a1d45a2c20efaea7c256f1ef"
server_key_version: "encryptionkey@0"
I20260812 07:07:29.037192 15926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:29.050773 15926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:29.051468 15926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:29.051728 15926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:29.052634 15926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:29.053939 15926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:29.054044 15926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:29.054150 15926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:29.054219 15926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:29.082700 15926 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.194:34801
I20260812 07:07:29.082749 16149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.194:34801 every 8 connection(s)
I20260812 07:07:29.083899 15926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-1/data/info.pb
I20260812 07:07:29.090497 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 15926
I20260812 07:07:29.090669 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-1/wal/instance
I20260812 07:07:29.091115 12035 external_mini_cluster.cc:1428] Setting key 963a39b8fc93a13c0281282af5a5c0b0
I20260812 07:07:29.093968 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.192.195:0
--local_ip_for_outbound_sockets=127.11.192.195
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43487
--builtin_ntp_servers=127.11.192.212:43953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:07:29.095830 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:29.086226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.194:41153 (local address 127.11.192.254:43487)
0812 07:07:29.086393 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:29.086398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:29.087639 (+ 1241us) server_negotiation.cc:407] Connection header received
0812 07:07:29.089327 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:29.089331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:29.089399 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:29.089483 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:29.090530 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.091310 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.093305 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.093688 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.093781 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:29.095198 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:29.095203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:29.095206 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:29.095224 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:29.095255 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:29.095266 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:29.095333 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:29.095542 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 07:07:29.095614 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:07:29.097036 16157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:29.086682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43487 (local address 127.11.192.194:41153)
0812 07:07:29.087507 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:29.087549 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:29.089112 (+ 1563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:29.089660 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:29.089668 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:29.089748 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:29.090369 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:29.090407 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.091459 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.091464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:29.093169 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:29.093185 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.093904 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.093913 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:29.094199 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:29.094874 (+ 675us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:07:29.094878 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:29.094925 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:07:29.095012 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:29.095349 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:29.095358 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:29.095366 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 07:07:29.095540 (+ 174us) client_negotiation.cc:240] Negotiation successful
0812 07:07:29.095878 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":259,"threads_started":1}
I20260812 07:07:29.098248 16150 heartbeater.cc:347] Connected to a master server at 127.11.192.254:43487
I20260812 07:07:29.098531 16150 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:29.099327 16150 heartbeater.cc:511] Master 127.11.192.254:43487 requested a full tablet report, sending...
I20260812 07:07:29.100688 15553 ts_manager.cc:194] Registered new tserver with Master: c5afefd62fa84c28b9c119728c7abbf8 (127.11.192.194:34801)
I20260812 07:07:29.101431 15553 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.192.194:41153
W20260812 07:07:29.422161 16158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:29.422401 16158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:29.422519 16158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:29.429348 16158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:29.429431 16158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:29.429575 16158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.192.195
I20260812 07:07:29.438129 16158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:43953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.192.195
--webserver_port=0
--tserver_master_addrs=127.11.192.254:43487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.192.195
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:29.439188 16158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:29.440908 16158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:29.453125 16165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:29.453147 16166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:29.453254 16170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:29.453780 16158 server_base.cc:1034] running on GCE node
I20260812 07:07:29.454341 16158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:29.455193 16158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:29.456390 16158 hybrid_clock.cc:648] HybridClock initialized: now 1786518449456348 us; error 31 us; skew 500 ppm
I20260812 07:07:29.456696 16158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:07:29.458734 16158 webserver.cc:460] Webserver started at http://127.11.192.195:45049/ using document root <none> and password file <none>
I20260812 07:07:29.459347 16158 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:29.459440 16158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:29.459741 16158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:29.462390 16158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-2/data/instance:
uuid: "288c0be196274e69bf8d9adb0fe7256d"
format_stamp: "Formatted at 2026-08-12 07:07:29 on dist-test-slave-lgr5"
server_key: "d40da3e6711ed4e456cc99968a03f7cb"
server_key_iv: "5d67f4025a4cb0dc13403d5faf262981"
server_key_version: "encryptionkey@0"
I20260812 07:07:29.463143 16158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance:
uuid: "288c0be196274e69bf8d9adb0fe7256d"
format_stamp: "Formatted at 2026-08-12 07:07:29 on dist-test-slave-lgr5"
server_key: "d40da3e6711ed4e456cc99968a03f7cb"
server_key_iv: "5d67f4025a4cb0dc13403d5faf262981"
server_key_version: "encryptionkey@0"
I20260812 07:07:29.467784 16158 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260812 07:07:29.471405 16177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:29.472597 16158 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:07:29.472741 16158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-2/wal
uuid: "288c0be196274e69bf8d9adb0fe7256d"
format_stamp: "Formatted at 2026-08-12 07:07:29 on dist-test-slave-lgr5"
server_key: "d40da3e6711ed4e456cc99968a03f7cb"
server_key_iv: "5d67f4025a4cb0dc13403d5faf262981"
server_key_version: "encryptionkey@0"
I20260812 07:07:29.472921 16158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:29.494899 16158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:29.495628 16158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:29.495895 16158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:29.496793 16158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:07:29.498133 16158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:07:29.498226 16158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:29.498324 16158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:07:29.498404 16158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:29.527516 16158 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.195:42281
I20260812 07:07:29.527576 16382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.195:42281 every 8 connection(s)
I20260812 07:07:29.528672 16158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-2/data/info.pb
I20260812 07:07:29.538542 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 16158
I20260812 07:07:29.538702 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-2/wal/instance
I20260812 07:07:29.539252 12035 external_mini_cluster.cc:1428] Setting key fe2789cc5b34fece7ce6b3bca029dde1
I20260812 07:07:29.540287 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:29.530958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.195:46753 (local address 127.11.192.254:43487)
0812 07:07:29.531141 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:29.531145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:29.532405 (+ 1260us) server_negotiation.cc:407] Connection header received
0812 07:07:29.534093 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:29.534098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:29.534178 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:29.534266 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:29.535296 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.536032 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.537526 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.537929 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.538039 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:29.539371 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:29.539377 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:29.539381 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:29.539398 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:29.539430 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:29.539442 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:29.539520 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:29.539789 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 07:07:29.539921 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:07:29.541234 16389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:29.531410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:43487 (local address 127.11.192.195:46753)
0812 07:07:29.532262 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:29.532307 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:29.533843 (+ 1536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:29.534485 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:29.534493 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:29.534579 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:07:29.535133 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:29.535151 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.536199 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.536203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:29.537381 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:29.537394 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.538146 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.538154 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:29.538372 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:29.539017 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:07:29.539021 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:07:29.539067 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:07:29.539170 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:07:29.539542 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:07:29.539553 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:07:29.539561 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 07:07:29.539744 (+ 183us) client_negotiation.cc:240] Negotiation successful
0812 07:07:29.540120 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":241,"threads_started":1}
I20260812 07:07:29.542484 16383 heartbeater.cc:347] Connected to a master server at 127.11.192.254:43487
I20260812 07:07:29.542737 16383 heartbeater.cc:464] Registering TS with master...
I20260812 07:07:29.543401 16383 heartbeater.cc:511] Master 127.11.192.254:43487 requested a full tablet report, sending...
I20260812 07:07:29.544714 15552 ts_manager.cc:194] Registered new tserver with Master: 288c0be196274e69bf8d9adb0fe7256d (127.11.192.195:42281)
I20260812 07:07:29.545567 15552 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.192.195:46753
I20260812 07:07:29.553951 12035 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:07:29.562700 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:29.557689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48086 (local address 127.11.192.254:43487)
0812 07:07:29.557833 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:29.557837 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:29.558298 (+ 461us) server_negotiation.cc:407] Connection header received
0812 07:07:29.558417 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:29.558420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:29.558494 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:29.558572 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:29.558972 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.559603 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.560900 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.561298 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.561385 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:29.562228 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:29.562231 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:29.562234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:29.562250 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:29.562273 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:29.562283 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:29.562341 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:29.562423 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 07:07:29.562498 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:07:29.566090 15552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:07:29.572762 15552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:07:29.593859 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:29.583815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:34801 (local address 127.0.0.1:52830)
0812 07:07:29.584452 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:29.584470 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:29.584589 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:29.585668 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:29.585677 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:29.585711 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:29.586085 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:29.586097 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.588170 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.588176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:29.590594 (+ 2418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:29.590612 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.592913 (+ 2301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.592920 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:29.593155 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:29.593207 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 07:07:29.593331 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 07:07:29.593522 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":534,"threads_started":1}
I20260812 07:07:29.595000 16415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:29.583958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52830 (local address 127.11.192.194:34801)
0812 07:07:29.584958 (+ 1000us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:29.584967 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:29.585002 (+ 35us) server_negotiation.cc:407] Connection header received
0812 07:07:29.585102 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:29.585109 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:29.585327 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:29.585640 (+ 313us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:29.586295 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.587951 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.590836 (+ 2885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.592355 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.592562 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:29.592779 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:29.594369 (+ 1590us) server_negotiation.cc:299] Negotiation successful
0812 07:07:29.594590 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":835,"thread_start_us":866,"threads_started":1}
I20260812 07:07:29.599146 16031 tablet_service.cc:1467] Processing CreateTablet for tablet a275b0d152ee49f8be26a40b20f8797c (DEFAULT_TABLE table=test-table [id=f5a297f0493f4d05a66d2bde80af1f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:29.600732 16031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a275b0d152ee49f8be26a40b20f8797c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:29.604471 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:29.585850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:45665 (local address 127.0.0.1:46028)
0812 07:07:29.594090 (+ 8240us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:29.594107 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:29.594192 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:29.598132 (+ 3940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:29.598137 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:29.598163 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:29.598316 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:29.598327 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.600170 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.600175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:29.602003 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:29.602018 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.603713 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.603719 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:29.603898 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:29.603933 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 07:07:29.604030 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 07:07:29.604192 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8132,"thread_start_us":15054,"threads_started":1}
I20260812 07:07:29.605005 16412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:29.583394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:42281 (local address 127.0.0.1:37654)
0812 07:07:29.584231 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:29.584268 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:29.584435 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:29.598753 (+ 14318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:29.598758 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:29.598783 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:29.599163 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:29.599175 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.601422 (+ 2247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.601428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:29.602945 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:29.602957 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.604407 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.604413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:29.604590 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:29.604621 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 07:07:29.604680 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 07:07:29.604786 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":784,"threads_started":1}
I20260812 07:07:29.605521 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:29.586243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46028 (local address 127.11.192.193:45665)
0812 07:07:29.597297 (+ 11054us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:29.597305 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:29.597336 (+ 31us) server_negotiation.cc:407] Connection header received
0812 07:07:29.597433 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:29.597440 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:29.597670 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:29.597855 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:29.598489 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.599960 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.602172 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.603537 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.603710 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:29.603901 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:29.604069 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 07:07:29.604257 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10908,"thread_start_us":10783,"threads_started":1}
I20260812 07:07:29.605954 16419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:29.584211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37654 (local address 127.11.192.195:42281)
0812 07:07:29.598025 (+ 13814us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:29.598041 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:29.598076 (+ 35us) server_negotiation.cc:407] Connection header received
0812 07:07:29.598162 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:29.598169 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:29.598550 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:29.599358 (+ 808us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:29.599620 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.601282 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.603115 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.604207 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.605253 (+ 1046us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:29.605435 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:29.605500 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 07:07:29.605653 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13340,"thread_start_us":13193,"threads_started":1}
I20260812 07:07:29.608781 16270 tablet_service.cc:1467] Processing CreateTablet for tablet a275b0d152ee49f8be26a40b20f8797c (DEFAULT_TABLE table=test-table [id=f5a297f0493f4d05a66d2bde80af1f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:29.609006 15788 tablet_service.cc:1467] Processing CreateTablet for tablet a275b0d152ee49f8be26a40b20f8797c (DEFAULT_TABLE table=test-table [id=f5a297f0493f4d05a66d2bde80af1f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:07:29.610041 16270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a275b0d152ee49f8be26a40b20f8797c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:29.610288 15788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a275b0d152ee49f8be26a40b20f8797c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:29.614845 16424 tablet_bootstrap.cc:492] T a275b0d152ee49f8be26a40b20f8797c P c5afefd62fa84c28b9c119728c7abbf8: Bootstrap starting.
I20260812 07:07:29.618368 16424 tablet_bootstrap.cc:654] T a275b0d152ee49f8be26a40b20f8797c P c5afefd62fa84c28b9c119728c7abbf8: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:29.619585 16424 log.cc:824] T a275b0d152ee49f8be26a40b20f8797c P c5afefd62fa84c28b9c119728c7abbf8: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:29.622408 16427 tablet_bootstrap.cc:492] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e: Bootstrap starting.
I20260812 07:07:29.622609 16424 tablet_bootstrap.cc:492] T a275b0d152ee49f8be26a40b20f8797c P c5afefd62fa84c28b9c119728c7abbf8: No bootstrap required, opened a new log
I20260812 07:07:29.622882 16424 ts_tablet_manager.cc:1397] T a275b0d152ee49f8be26a40b20f8797c P c5afefd62fa84c28b9c119728c7abbf8: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 07:07:29.625088 16427 tablet_bootstrap.cc:654] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:29.625969 16427 log.cc:824] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:29.629215 16427 tablet_bootstrap.cc:492] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e: No bootstrap required, opened a new log
I20260812 07:07:29.629434 16427 ts_tablet_manager.cc:1397] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 07:07:29.631382 16424 raft_consensus.cc:348] T a275b0d152ee49f8be26a40b20f8797c P c5afefd62fa84c28b9c119728c7abbf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "288c0be196274e69bf8d9adb0fe7256d" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42281 } } peers { permanent_uuid: "c5afefd62fa84c28b9c119728c7abbf8" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34801 } } peers { permanent_uuid: "f8fb4ebb1c4c452bb77af1aeaf2b0d5e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45665 } }
I20260812 07:07:29.631690 16424 raft_consensus.cc:374] T a275b0d152ee49f8be26a40b20f8797c P c5afefd62fa84c28b9c119728c7abbf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:29.631788 16424 raft_consensus.cc:729] T a275b0d152ee49f8be26a40b20f8797c P c5afefd62fa84c28b9c119728c7abbf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5afefd62fa84c28b9c119728c7abbf8, State: Initialized, Role: FOLLOWER
I20260812 07:07:29.631891 16428 tablet_bootstrap.cc:492] T a275b0d152ee49f8be26a40b20f8797c P 288c0be196274e69bf8d9adb0fe7256d: Bootstrap starting.
I20260812 07:07:29.632360 16424 consensus_queue.cc:260] T a275b0d152ee49f8be26a40b20f8797c P c5afefd62fa84c28b9c119728c7abbf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "288c0be196274e69bf8d9adb0fe7256d" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42281 } } peers { permanent_uuid: "c5afefd62fa84c28b9c119728c7abbf8" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34801 } } peers { permanent_uuid: "f8fb4ebb1c4c452bb77af1aeaf2b0d5e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45665 } }
I20260812 07:07:29.635179 16428 tablet_bootstrap.cc:654] T a275b0d152ee49f8be26a40b20f8797c P 288c0be196274e69bf8d9adb0fe7256d: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:29.636309 16428 log.cc:824] T a275b0d152ee49f8be26a40b20f8797c P 288c0be196274e69bf8d9adb0fe7256d: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:29.636334 16150 heartbeater.cc:503] Master 127.11.192.254:43487 was elected leader, sending a full tablet report...
I20260812 07:07:29.636950 16424 ts_tablet_manager.cc:1428] T a275b0d152ee49f8be26a40b20f8797c P c5afefd62fa84c28b9c119728c7abbf8: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260812 07:07:29.638029 16427 raft_consensus.cc:348] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "288c0be196274e69bf8d9adb0fe7256d" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42281 } } peers { permanent_uuid: "c5afefd62fa84c28b9c119728c7abbf8" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34801 } } peers { permanent_uuid: "f8fb4ebb1c4c452bb77af1aeaf2b0d5e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45665 } }
I20260812 07:07:29.638423 16427 raft_consensus.cc:374] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:29.638533 16427 raft_consensus.cc:729] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8fb4ebb1c4c452bb77af1aeaf2b0d5e, State: Initialized, Role: FOLLOWER
I20260812 07:07:29.639134 16427 consensus_queue.cc:260] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "288c0be196274e69bf8d9adb0fe7256d" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42281 } } peers { permanent_uuid: "c5afefd62fa84c28b9c119728c7abbf8" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34801 } } peers { permanent_uuid: "f8fb4ebb1c4c452bb77af1aeaf2b0d5e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45665 } }
I20260812 07:07:29.640005 16428 tablet_bootstrap.cc:492] T a275b0d152ee49f8be26a40b20f8797c P 288c0be196274e69bf8d9adb0fe7256d: No bootstrap required, opened a new log
I20260812 07:07:29.640233 15917 heartbeater.cc:503] Master 127.11.192.254:43487 was elected leader, sending a full tablet report...
I20260812 07:07:29.640278 16428 ts_tablet_manager.cc:1397] T a275b0d152ee49f8be26a40b20f8797c P 288c0be196274e69bf8d9adb0fe7256d: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.003s
I20260812 07:07:29.642045 16427 ts_tablet_manager.cc:1428] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260812 07:07:29.648102 16428 raft_consensus.cc:348] T a275b0d152ee49f8be26a40b20f8797c P 288c0be196274e69bf8d9adb0fe7256d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "288c0be196274e69bf8d9adb0fe7256d" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42281 } } peers { permanent_uuid: "c5afefd62fa84c28b9c119728c7abbf8" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34801 } } peers { permanent_uuid: "f8fb4ebb1c4c452bb77af1aeaf2b0d5e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45665 } }
I20260812 07:07:29.648424 16428 raft_consensus.cc:374] T a275b0d152ee49f8be26a40b20f8797c P 288c0be196274e69bf8d9adb0fe7256d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:29.648516 16428 raft_consensus.cc:729] T a275b0d152ee49f8be26a40b20f8797c P 288c0be196274e69bf8d9adb0fe7256d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 288c0be196274e69bf8d9adb0fe7256d, State: Initialized, Role: FOLLOWER
I20260812 07:07:29.649132 16428 consensus_queue.cc:260] T a275b0d152ee49f8be26a40b20f8797c P 288c0be196274e69bf8d9adb0fe7256d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "288c0be196274e69bf8d9adb0fe7256d" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42281 } } peers { permanent_uuid: "c5afefd62fa84c28b9c119728c7abbf8" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34801 } } peers { permanent_uuid: "f8fb4ebb1c4c452bb77af1aeaf2b0d5e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45665 } }
I20260812 07:07:29.650139 16383 heartbeater.cc:503] Master 127.11.192.254:43487 was elected leader, sending a full tablet report...
I20260812 07:07:29.650753 16428 ts_tablet_manager.cc:1428] T a275b0d152ee49f8be26a40b20f8797c P 288c0be196274e69bf8d9adb0fe7256d: Time spent starting tablet: real 0.010s user 0.006s sys 0.004s
W20260812 07:07:29.652659 15920 tablet.cc:2369] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:07:29.667274 16435 raft_consensus.cc:482] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:07:29.667467 16435 raft_consensus.cc:504] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "288c0be196274e69bf8d9adb0fe7256d" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42281 } } peers { permanent_uuid: "c5afefd62fa84c28b9c119728c7abbf8" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34801 } } peers { permanent_uuid: "f8fb4ebb1c4c452bb77af1aeaf2b0d5e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45665 } }
I20260812 07:07:29.668881 16435 leader_election.cc:288] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 288c0be196274e69bf8d9adb0fe7256d (127.11.192.195:42281), c5afefd62fa84c28b9c119728c7abbf8 (127.11.192.194:34801)
I20260812 07:07:29.675640 16419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:29.669333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:49437 (local address 127.11.192.195:42281)
0812 07:07:29.669523 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:29.669529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:29.670128 (+ 599us) server_negotiation.cc:407] Connection header received
0812 07:07:29.670309 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:29.670313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:29.670374 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:29.670453 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:29.671109 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.671820 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.673919 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.674777 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.674943 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:29.675017 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:29.675308 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 07:07:29.675416 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:07:29.675698 16415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:29.669205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.192.193:59579 (local address 127.11.192.194:34801)
0812 07:07:29.669393 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:29.669399 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:29.670103 (+ 704us) server_negotiation.cc:407] Connection header received
0812 07:07:29.670281 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:29.670286 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:29.670362 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:29.670453 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:07:29.671195 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.671899 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.673996 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.674815 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.674968 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:29.675062 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:29.675349 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 07:07:29.675470 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:07:29.675833 16440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:29.669177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.194:34801 (local address 127.11.192.193:59579)
0812 07:07:29.669985 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:29.670009 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:29.670154 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:29.670478 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:29.670483 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:29.670538 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:29.671038 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:29.671054 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.672046 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.672050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:29.673747 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:29.673762 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.675008 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.675016 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:29.675194 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:29.675217 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 07:07:29.675294 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 07:07:29.675411 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"mutex_wait_us":48,"spinlock_wait_cycles":27392,"thread_start_us":393,"threads_started":1}
I20260812 07:07:29.675841 16441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:29.669275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.195:42281 (local address 127.11.192.193:49437)
0812 07:07:29.670043 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:29.670056 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:07:29.670207 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:07:29.670484 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:07:29.670488 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:07:29.670511 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:07:29.670904 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:29.670919 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.671956 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.671961 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:29.673709 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:07:29.673726 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.674956 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.674963 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:07:29.675153 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:29.675177 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 07:07:29.675262 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 07:07:29.675381 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"mutex_wait_us":60,"spinlock_wait_cycles":16256,"thread_start_us":480,"threads_started":1}
I20260812 07:07:29.676802 16061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a275b0d152ee49f8be26a40b20f8797c" candidate_uuid: "f8fb4ebb1c4c452bb77af1aeaf2b0d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5afefd62fa84c28b9c119728c7abbf8" is_pre_election: true
I20260812 07:07:29.676791 16304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a275b0d152ee49f8be26a40b20f8797c" candidate_uuid: "f8fb4ebb1c4c452bb77af1aeaf2b0d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "288c0be196274e69bf8d9adb0fe7256d" is_pre_election: true
I20260812 07:07:29.677342 16304 raft_consensus.cc:2454] T a275b0d152ee49f8be26a40b20f8797c P 288c0be196274e69bf8d9adb0fe7256d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8fb4ebb1c4c452bb77af1aeaf2b0d5e in term 0.
I20260812 07:07:29.677342 16061 raft_consensus.cc:2454] T a275b0d152ee49f8be26a40b20f8797c P c5afefd62fa84c28b9c119728c7abbf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8fb4ebb1c4c452bb77af1aeaf2b0d5e in term 0.
I20260812 07:07:29.677974 15693 leader_election.cc:302] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 288c0be196274e69bf8d9adb0fe7256d, f8fb4ebb1c4c452bb77af1aeaf2b0d5e; no voters:
I20260812 07:07:29.678335 16435 raft_consensus.cc:2781] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:07:29.678460 16435 raft_consensus.cc:482] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:07:29.678556 16435 raft_consensus.cc:3037] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:29.680648 16435 raft_consensus.cc:504] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "288c0be196274e69bf8d9adb0fe7256d" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42281 } } peers { permanent_uuid: "c5afefd62fa84c28b9c119728c7abbf8" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34801 } } peers { permanent_uuid: "f8fb4ebb1c4c452bb77af1aeaf2b0d5e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45665 } }
I20260812 07:07:29.681172 16435 leader_election.cc:288] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e [CANDIDATE]: Term 1 election: Requested vote from peers 288c0be196274e69bf8d9adb0fe7256d (127.11.192.195:42281), c5afefd62fa84c28b9c119728c7abbf8 (127.11.192.194:34801)
I20260812 07:07:29.681560 16304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a275b0d152ee49f8be26a40b20f8797c" candidate_uuid: "f8fb4ebb1c4c452bb77af1aeaf2b0d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "288c0be196274e69bf8d9adb0fe7256d"
I20260812 07:07:29.681569 16061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a275b0d152ee49f8be26a40b20f8797c" candidate_uuid: "f8fb4ebb1c4c452bb77af1aeaf2b0d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5afefd62fa84c28b9c119728c7abbf8"
I20260812 07:07:29.681715 16304 raft_consensus.cc:3037] T a275b0d152ee49f8be26a40b20f8797c P 288c0be196274e69bf8d9adb0fe7256d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:29.681717 16061 raft_consensus.cc:3037] T a275b0d152ee49f8be26a40b20f8797c P c5afefd62fa84c28b9c119728c7abbf8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:29.683730 16304 raft_consensus.cc:2454] T a275b0d152ee49f8be26a40b20f8797c P 288c0be196274e69bf8d9adb0fe7256d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8fb4ebb1c4c452bb77af1aeaf2b0d5e in term 1.
I20260812 07:07:29.683838 16061 raft_consensus.cc:2454] T a275b0d152ee49f8be26a40b20f8797c P c5afefd62fa84c28b9c119728c7abbf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8fb4ebb1c4c452bb77af1aeaf2b0d5e in term 1.
I20260812 07:07:29.684098 15693 leader_election.cc:302] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 288c0be196274e69bf8d9adb0fe7256d, f8fb4ebb1c4c452bb77af1aeaf2b0d5e; no voters:
I20260812 07:07:29.684420 16435 raft_consensus.cc:2781] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:29.684813 16435 raft_consensus.cc:686] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e [term 1 LEADER]: Becoming Leader. State: Replica: f8fb4ebb1c4c452bb77af1aeaf2b0d5e, State: Running, Role: LEADER
I20260812 07:07:29.685258 16435 consensus_queue.cc:237] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "288c0be196274e69bf8d9adb0fe7256d" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42281 } } peers { permanent_uuid: "c5afefd62fa84c28b9c119728c7abbf8" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34801 } } peers { permanent_uuid: "f8fb4ebb1c4c452bb77af1aeaf2b0d5e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45665 } }
I20260812 07:07:29.689121 15553 catalog_manager.cc:5495] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e reported cstate change: term changed from 0 to 1, leader changed from <none> to f8fb4ebb1c4c452bb77af1aeaf2b0d5e (127.11.192.193). New cstate: current_term: 1 leader_uuid: "f8fb4ebb1c4c452bb77af1aeaf2b0d5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "288c0be196274e69bf8d9adb0fe7256d" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5afefd62fa84c28b9c119728c7abbf8" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8fb4ebb1c4c452bb77af1aeaf2b0d5e" member_type: VOTER last_known_addr { host: "127.11.192.193" port: 45665 } health_report { overall_health: HEALTHY } } }
I20260812 07:07:29.703781 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:29.698775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46044 (local address 127.11.192.193:45665)
0812 07:07:29.698960 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:29.698966 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:29.699053 (+ 87us) server_negotiation.cc:407] Connection header received
0812 07:07:29.699246 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:29.699250 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:29.699337 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:29.699455 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:07:29.699811 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.700507 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.701827 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.702249 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.702356 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:29.702795 (+ 439us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:07:29.703242 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:07:29.703368 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:29.703473 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 07:07:29.703546 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 07:07:29.711206 16304 raft_consensus.cc:1261] T a275b0d152ee49f8be26a40b20f8797c P 288c0be196274e69bf8d9adb0fe7256d [term 1 FOLLOWER]: Refusing update from remote peer f8fb4ebb1c4c452bb77af1aeaf2b0d5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:07:29.711206 16061 raft_consensus.cc:1261] T a275b0d152ee49f8be26a40b20f8797c P c5afefd62fa84c28b9c119728c7abbf8 [term 1 FOLLOWER]: Refusing update from remote peer f8fb4ebb1c4c452bb77af1aeaf2b0d5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:07:29.712218 16435 consensus_queue.cc:1035] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "288c0be196274e69bf8d9adb0fe7256d" member_type: VOTER last_known_addr { host: "127.11.192.195" port: 42281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:07:29.712495 16442 consensus_queue.cc:1035] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5afefd62fa84c28b9c119728c7abbf8" member_type: VOTER last_known_addr { host: "127.11.192.194" port: 34801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:07:29.724145 16450 mvcc.cc:205] Tried to move back new op lower bound from 7317579570007838720 to 7317579569914085376. Current Snapshot: MvccSnapshot[applied={T|T < 7317579570007838720}]
I20260812 07:07:29.725445 16451 mvcc.cc:205] Tried to move back new op lower bound from 7317579570007838720 to 7317579569914085376. Current Snapshot: MvccSnapshot[applied={T|T < 7317579570007838720}]
I20260812 07:07:29.734982 15549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:07:29.735265 15549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:07:29.738883 15549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bbceae5efa64427fb346727afed6bcf9: Sending DeleteTablet for 3 replicas of tablet a275b0d152ee49f8be26a40b20f8797c
I20260812 07:07:29.739909 16031 tablet_service.cc:1514] Processing DeleteTablet for tablet a275b0d152ee49f8be26a40b20f8797c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:29 UTC) from {username='slave'} at 127.0.0.1:52830
I20260812 07:07:29.739967 16270 tablet_service.cc:1514] Processing DeleteTablet for tablet a275b0d152ee49f8be26a40b20f8797c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:29 UTC) from {username='slave'} at 127.0.0.1:37654
I20260812 07:07:29.740334 15788 tablet_service.cc:1514] Processing DeleteTablet for tablet a275b0d152ee49f8be26a40b20f8797c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:07:29 UTC) from {username='slave'} at 127.0.0.1:46028
I20260812 07:07:29.746557 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 15666
I20260812 07:07:29.750818 16460 tablet_replica.cc:330] T a275b0d152ee49f8be26a40b20f8797c P 288c0be196274e69bf8d9adb0fe7256d: stopping tablet replica
I20260812 07:07:29.751228 16460 raft_consensus.cc:2229] T a275b0d152ee49f8be26a40b20f8797c P 288c0be196274e69bf8d9adb0fe7256d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:07:29.751715 16460 raft_consensus.cc:2258] T a275b0d152ee49f8be26a40b20f8797c P 288c0be196274e69bf8d9adb0fe7256d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:29.753683 16460 ts_tablet_manager.cc:1905] T a275b0d152ee49f8be26a40b20f8797c P 288c0be196274e69bf8d9adb0fe7256d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:29.760522 16460 ts_tablet_manager.cc:1918] T a275b0d152ee49f8be26a40b20f8797c P 288c0be196274e69bf8d9adb0fe7256d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:07:29.760687 16460 log.cc:1196] T a275b0d152ee49f8be26a40b20f8797c P 288c0be196274e69bf8d9adb0fe7256d: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-2/wal/wals/a275b0d152ee49f8be26a40b20f8797c
I20260812 07:07:29.761221 16460 ts_tablet_manager.cc:1939] T a275b0d152ee49f8be26a40b20f8797c P 288c0be196274e69bf8d9adb0fe7256d: Deleting consensus metadata
I20260812 07:07:29.762967 15529 catalog_manager.cc:4841] TS 288c0be196274e69bf8d9adb0fe7256d (127.11.192.195:42281): tablet a275b0d152ee49f8be26a40b20f8797c (table test-table [id=f5a297f0493f4d05a66d2bde80af1f6e]) successfully deleted
I20260812 07:07:29.764609 16463 tablet_replica.cc:330] T a275b0d152ee49f8be26a40b20f8797c P c5afefd62fa84c28b9c119728c7abbf8: stopping tablet replica
I20260812 07:07:29.765120 16463 raft_consensus.cc:2229] T a275b0d152ee49f8be26a40b20f8797c P c5afefd62fa84c28b9c119728c7abbf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:07:29.765790 16462 tablet_replica.cc:330] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e: stopping tablet replica
I20260812 07:07:29.766278 16463 raft_consensus.cc:2258] T a275b0d152ee49f8be26a40b20f8797c P c5afefd62fa84c28b9c119728c7abbf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:29.766434 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:29.747811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46054 (local address 127.11.192.193:45665)
0812 07:07:29.747981 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:29.747987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:29.757991 (+ 10004us) server_negotiation.cc:407] Connection header received
0812 07:07:29.758385 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:29.758392 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:29.758499 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:29.758569 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:29.759214 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.760427 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.762488 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.763193 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.763289 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:29.764844 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:29.764855 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:29.764871 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:29.765625 (+ 754us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:29.765685 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:29.765709 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:29.765821 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:29.765963 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 07:07:29.766066 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:07:29.767289 15907 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:29.767287 16462 raft_consensus.cc:2229] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:07:29.768333 16463 ts_tablet_manager.cc:1905] T a275b0d152ee49f8be26a40b20f8797c P c5afefd62fa84c28b9c119728c7abbf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:07:29.775079 16463 ts_tablet_manager.cc:1918] T a275b0d152ee49f8be26a40b20f8797c P c5afefd62fa84c28b9c119728c7abbf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:07:29.775241 16463 log.cc:1196] T a275b0d152ee49f8be26a40b20f8797c P c5afefd62fa84c28b9c119728c7abbf8: Deleting WAL directory at /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518136653111-12035-0/minicluster-data/ts-1/wal/wals/a275b0d152ee49f8be26a40b20f8797c
I20260812 07:07:29.775777 16463 ts_tablet_manager.cc:1939] T a275b0d152ee49f8be26a40b20f8797c P c5afefd62fa84c28b9c119728c7abbf8: Deleting consensus metadata
I20260812 07:07:29.777566 15535 catalog_manager.cc:4841] TS c5afefd62fa84c28b9c119728c7abbf8 (127.11.192.194:34801): tablet a275b0d152ee49f8be26a40b20f8797c (table test-table [id=f5a297f0493f4d05a66d2bde80af1f6e]) successfully deleted
I20260812 07:07:29.864112 16462 raft_consensus.cc:2258] T a275b0d152ee49f8be26a40b20f8797c P f8fb4ebb1c4c452bb77af1aeaf2b0d5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:07:29.864681 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 15666
W20260812 07:07:29.890128 15529 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.192.193:45665 (error 108)
I20260812 07:07:29.890326 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 15926
I20260812 07:07:29.892074 16412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:29.891556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:45665 (local address 127.0.0.1:46056)
0812 07:07:29.891746 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:29.891889 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:45665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 07:07:29.892365 15529 catalog_manager.cc:4584] TS f8fb4ebb1c4c452bb77af1aeaf2b0d5e (127.11.192.193:45665): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a275b0d152ee49f8be26a40b20f8797c: Network error: Client connection negotiation failed: client connection to 127.11.192.193:45665: connect: Connection refused (error 111)
I20260812 07:07:29.897001 16415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:29.891114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52838 (local address 127.11.192.194:34801)
0812 07:07:29.891270 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:29.891275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:29.891293 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:07:29.891513 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:29.891518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:29.891626 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:29.891731 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:29.892272 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.893060 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.894263 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:29.894650 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:29.894736 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:29.895651 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:29.895658 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:29.895671 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:29.896350 (+ 679us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:29.896397 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:29.896418 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:29.896500 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:29.896612 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 07:07:29.896705 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 07:07:29.897718 16144 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:29.945935 16412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:29.945444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:45665 (local address 127.0.0.1:46060)
0812 07:07:29.945627 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:29.945748 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:45665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 07:07:29.947074 16412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:29.946664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:45665 (local address 127.0.0.1:46072)
0812 07:07:29.946862 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:29.946938 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:45665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 07:07:29.985847 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 15926
I20260812 07:07:30.007599 16412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:30.007114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:45665 (local address 127.0.0.1:46086)
0812 07:07:30.007343 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:30.007433 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:45665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 07:07:30.008687 16412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:30.008333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:45665 (local address 127.0.0.1:46102)
0812 07:07:30.008487 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:30.008557 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:45665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 07:07:30.010573 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 16158
I20260812 07:07:30.016782 16419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:30.011235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37668 (local address 127.11.192.195:42281)
0812 07:07:30.011409 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:30.011414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:30.011433 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:07:30.011557 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:30.011561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:30.011646 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:30.011726 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:30.012053 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:30.012802 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:30.014069 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:30.014446 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:30.014540 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:30.015461 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:30.015469 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:30.015494 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:30.016101 (+ 607us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:30.016164 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:30.016189 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:30.016290 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:30.016393 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 07:07:30.016489 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:07:30.017385 16372 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:30.089143 16412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:30.088661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:45665 (local address 127.0.0.1:46104)
0812 07:07:30.088862 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:30.088948 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:45665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 07:07:30.090139 16412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:30.089750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.193:45665 (local address 127.0.0.1:46114)
0812 07:07:30.089952 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 07:07:30.090028 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.193:45665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260812 07:07:30.103621 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 16158
I20260812 07:07:30.132279 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 15506
I20260812 07:07:30.133296 15599 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:30.239158 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 15506
2026-08-12T07:07:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2636 ms)
[----------] 6 tests from EncryptionPolicyTest (21173 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:07:30 dist-test-slave-lgr5 krb5kdc[16475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:07:30 dist-test-slave-lgr5 krb5kdc[16475](info): set up 2 sockets
Aug 12 07:07:30 dist-test-slave-lgr5 krb5kdc[16475](info): commencing operation
krb5kdc: starting...
W20260812 07:07:31.503191 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.005s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:07:31 dist-test-slave-lgr5 krb5kdc[16475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:07:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:07:31.753196 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33671
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:32.084604 16500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:32.084867 16500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:32.084997 16500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:32.091807 16500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:32.091902 16500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:32.091971 16500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:32.092049 16500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:32.092126 16500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:32.101300 16500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33671
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:32.102445 16500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:32.103654 16500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:32.114362 16508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:32.114396 16509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:32.114591 16514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:32.115193 16500 server_base.cc:1034] running on GCE node
I20260812 07:07:32.115922 16500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:32.117208 16500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:32.118407 16500 hybrid_clock.cc:648] HybridClock initialized: now 1786518452118358 us; error 34 us; skew 500 ppm
I20260812 07:07:32.118733 16500 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:32 dist-test-slave-lgr5 krb5kdc[16475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518452, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:32.121604 16500 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:07:32.122898 16500 webserver.cc:460] Webserver started at http://127.0.0.1:41267/ using document root <none> and password file <none>
I20260812 07:07:32.123517 16500 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:32.123636 16500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:32.123920 16500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:32.126772 16500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "4668a952dd15459888fdaf9052ffb110"
format_stamp: "Formatted at 2026-08-12 07:07:32 on dist-test-slave-lgr5"
server_key: "c8ec4c30407ab30fb4aef2e2c62bf43b"
server_key_iv: "9c012bd206a3613a09d70a1f345904af"
server_key_version: "encryptionkey@0"
I20260812 07:07:32.127512 16500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "4668a952dd15459888fdaf9052ffb110"
format_stamp: "Formatted at 2026-08-12 07:07:32 on dist-test-slave-lgr5"
server_key: "c8ec4c30407ab30fb4aef2e2c62bf43b"
server_key_iv: "9c012bd206a3613a09d70a1f345904af"
server_key_version: "encryptionkey@0"
I20260812 07:07:32.132230 16500 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260812 07:07:32.135345 16530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:32.136562 16500 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:07:32.136729 16500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "4668a952dd15459888fdaf9052ffb110"
format_stamp: "Formatted at 2026-08-12 07:07:32 on dist-test-slave-lgr5"
server_key: "c8ec4c30407ab30fb4aef2e2c62bf43b"
server_key_iv: "9c012bd206a3613a09d70a1f345904af"
server_key_version: "encryptionkey@0"
I20260812 07:07:32.136895 16500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:32.159984 16500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:32.163827 16500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:32.164117 16500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:32.182214 16500 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33671
I20260812 07:07:32.182235 16616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33671 every 8 connection(s)
I20260812 07:07:32.183547 16500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:32.185447 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 16500
I20260812 07:07:32.185643 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:07:32.186322 12035 external_mini_cluster.cc:1428] Setting key e2c6661a6a5099259e84d8c8ec01de11
I20260812 07:07:32.187249 16617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:32.200752 16617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110: Bootstrap starting.
Aug 12 07:07:32 dist-test-slave-lgr5 krb5kdc[16475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 07:07:32.204384 16617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:32.205681 16617 log.cc:824] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:32.208354 16617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110: No bootstrap required, opened a new log
I20260812 07:07:32.209898 16623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:32.188279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53603 (local address 127.0.0.1:33671)
0812 07:07:32.189089 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:32.189107 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:32.189158 (+ 51us) server_negotiation.cc:407] Connection header received
0812 07:07:32.190774 (+ 1616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:32.190800 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:32.191286 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:32.191701 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:32.193138 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:32.195126 (+ 1988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:32.196468 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:32.197473 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:32.197639 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:32.202051 (+ 4412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:32.202068 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:32.202086 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:32.202146 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:32.205931 (+ 3785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:32.206724 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:32.206731 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:32.206737 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:32.206902 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:32.207356 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:32.207360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:32.207362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:32.207854 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:32.208173 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:32.208694 (+ 521us) server_negotiation.cc:299] Negotiation successful
0812 07:07:32.208996 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":330,"threads_started":1}
I20260812 07:07:32.213519 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:07:32.217274 16617 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4668a952dd15459888fdaf9052ffb110" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33671 } }
I20260812 07:07:32.217711 16617 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:32.217829 16617 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4668a952dd15459888fdaf9052ffb110, State: Initialized, Role: FOLLOWER
I20260812 07:07:32.218431 16617 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4668a952dd15459888fdaf9052ffb110" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33671 } }
I20260812 07:07:32.218716 16617 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:32.218835 16617 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:32.219058 16617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:32.222064 16617 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4668a952dd15459888fdaf9052ffb110" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33671 } }
I20260812 07:07:32.222522 16617 leader_election.cc:302] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4668a952dd15459888fdaf9052ffb110; no voters:
I20260812 07:07:32.223119 16617 leader_election.cc:288] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:32.223209 16643 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:32.225288 16617 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:32.229851 16643 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110 [term 1 LEADER]: Becoming Leader. State: Replica: 4668a952dd15459888fdaf9052ffb110, State: Running, Role: LEADER
I20260812 07:07:32.230535 16643 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4668a952dd15459888fdaf9052ffb110" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33671 } }
I20260812 07:07:32.233549 16644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4668a952dd15459888fdaf9052ffb110" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4668a952dd15459888fdaf9052ffb110" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33671 } } }
I20260812 07:07:32.233619 16645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4668a952dd15459888fdaf9052ffb110. Latest consensus state: current_term: 1 leader_uuid: "4668a952dd15459888fdaf9052ffb110" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4668a952dd15459888fdaf9052ffb110" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33671 } } }
I20260812 07:07:32.234017 16644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:32.234047 16645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:32.235622 16655 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:32.237126 16623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:32.219157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47411 (local address 127.0.0.1:33671)
0812 07:07:32.219298 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:32.219302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:32.219867 (+ 565us) server_negotiation.cc:407] Connection header received
0812 07:07:32.220011 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:32.220014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:32.220073 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:32.220185 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:32.221908 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:32.222803 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:32.224652 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:32.225162 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:32.225288 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:32.227808 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:32.227813 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:32.227817 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:32.227849 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:32.231069 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:32.231831 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:32.231837 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:32.231839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:32.231956 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:32.235319 (+ 3363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:32.235325 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:32.235329 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:32.235751 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:32.236053 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:32.236589 (+ 536us) server_negotiation.cc:299] Negotiation successful
0812 07:07:32.236770 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 07:07:32.240866 16655 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:32.250087 16655 catalog_manager.cc:1347] Generated new cluster ID: b0662fd3f7b34efea74e3a7b4c13b552
I20260812 07:07:32.250195 16655 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:32.259518 16655 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:32.260741 16655 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:32.266328 16655 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4668a952dd15459888fdaf9052ffb110: Generated new TSK 0
I20260812 07:07:32.266963 16655 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:07:32.277797 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 16500
I20260812 07:07:32.278884 16611 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:32.366549 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 16500
W20260812 07:07:32.386826 16472 connection.cc:535] client connection to 127.0.0.1:33671 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33671): Connection reset by peer (error 104)
2026-08-12T07:07:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2121 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:07:32 dist-test-slave-lgr5 krb5kdc[16678](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:07:32 dist-test-slave-lgr5 krb5kdc[16678](info): set up 2 sockets
Aug 12 07:07:32 dist-test-slave-lgr5 krb5kdc[16678](info): commencing operation
krb5kdc: starting...
W20260812 07:07:33.610909 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:07:33 dist-test-slave-lgr5 krb5kdc[16678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:07:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:07:33.864913 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43479
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:34.188715 16700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:34.188972 16700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:34.189102 16700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:34.196105 16700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:34.196202 16700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:34.196275 16700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:34.196363 16700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:34.196421 16700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:34.205786 16700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43479
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:34.206936 16700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:34.208226 16700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:34.218926 16713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:34.219062 16717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:34.219022 16714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:34.220000 16700 server_base.cc:1034] running on GCE node
I20260812 07:07:34.220706 16700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:34.221932 16700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:34.223126 16700 hybrid_clock.cc:648] HybridClock initialized: now 1786518454223086 us; error 28 us; skew 500 ppm
I20260812 07:07:34.223431 16700 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:34 dist-test-slave-lgr5 krb5kdc[16678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518454, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:34.229421 16700 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:07:34.230705 16700 webserver.cc:460] Webserver started at http://127.0.0.1:34777/ using document root <none> and password file <none>
I20260812 07:07:34.231320 16700 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:34.231431 16700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:34.231707 16700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:34.234537 16700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "bfbdddaf01ba43de83d08e7bfd20f033"
format_stamp: "Formatted at 2026-08-12 07:07:34 on dist-test-slave-lgr5"
server_key: "01a522c743165603f3dd77c65e2bde95"
server_key_iv: "71412ef9369c039f84050bb9156709bf"
server_key_version: "encryptionkey@0"
I20260812 07:07:34.235265 16700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "bfbdddaf01ba43de83d08e7bfd20f033"
format_stamp: "Formatted at 2026-08-12 07:07:34 on dist-test-slave-lgr5"
server_key: "01a522c743165603f3dd77c65e2bde95"
server_key_iv: "71412ef9369c039f84050bb9156709bf"
server_key_version: "encryptionkey@0"
I20260812 07:07:34.240007 16700 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260812 07:07:34.243151 16729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:34.244374 16700 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260812 07:07:34.244531 16700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "bfbdddaf01ba43de83d08e7bfd20f033"
format_stamp: "Formatted at 2026-08-12 07:07:34 on dist-test-slave-lgr5"
server_key: "01a522c743165603f3dd77c65e2bde95"
server_key_iv: "71412ef9369c039f84050bb9156709bf"
server_key_version: "encryptionkey@0"
I20260812 07:07:34.244695 16700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:34.258474 16700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:34.262430 16700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:34.262676 16700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:34.279436 16700 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43479
I20260812 07:07:34.279491 16815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43479 every 8 connection(s)
I20260812 07:07:34.280719 16700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:34.284291 16817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:34.289470 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 16700
I20260812 07:07:34.289656 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:07:34.290191 12035 external_mini_cluster.cc:1428] Setting key 2b8f08ed693c7c29d9f75dec7401f4bf
I20260812 07:07:34.294788 16817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033: Bootstrap starting.
I20260812 07:07:34.298204 16817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:34.299315 16817 log.cc:824] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:34.301862 16817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033: No bootstrap required, opened a new log
I20260812 07:07:34.302767 16821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:34.291882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60529 (local address 127.0.0.1:43479)
0812 07:07:34.292721 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:34.292734 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:34.292775 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:07:34.294385 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:34.294408 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:34.294778 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:34.295106 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:34.295682 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:34.296989 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:34.298751 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:34.299392 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:34.299577 (+ 185us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:34.300728 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:34.300735 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:34.300748 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:34.301219 (+ 471us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:34.301284 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:34.301313 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:34.301444 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:34.301708 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 07:07:34.301849 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":402,"threads_started":1}
I20260812 07:07:34.306224 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:07:34.309422 16817 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfbdddaf01ba43de83d08e7bfd20f033" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43479 } }
I20260812 07:07:34.309736 16817 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:34.309820 16817 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfbdddaf01ba43de83d08e7bfd20f033, State: Initialized, Role: FOLLOWER
I20260812 07:07:34.310479 16817 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfbdddaf01ba43de83d08e7bfd20f033" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43479 } }
I20260812 07:07:34.310702 16817 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:34.310835 16817 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:34.311021 16817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:34.313230 16817 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfbdddaf01ba43de83d08e7bfd20f033" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43479 } }
I20260812 07:07:34.313673 16817 leader_election.cc:302] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfbdddaf01ba43de83d08e7bfd20f033; no voters:
I20260812 07:07:34.314254 16817 leader_election.cc:288] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:34.314342 16836 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:34.316277 16821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:34.310160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53817 (local address 127.0.0.1:43479)
0812 07:07:34.310334 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:34.310339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:34.310795 (+ 456us) server_negotiation.cc:407] Connection header received
0812 07:07:34.310959 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:34.310968 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:34.311029 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:34.311103 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:34.311594 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:34.312430 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:34.313747 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:34.314254 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:34.314363 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:34.315508 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:34.315514 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:34.315519 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:34.315538 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:34.315592 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:34.315608 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:34.315694 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:34.315896 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 07:07:34.315944 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 07:07:34.316426 16817 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:34.321180 16836 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033 [term 1 LEADER]: Becoming Leader. State: Replica: bfbdddaf01ba43de83d08e7bfd20f033, State: Running, Role: LEADER
I20260812 07:07:34.321734 16836 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfbdddaf01ba43de83d08e7bfd20f033" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43479 } }
I20260812 07:07:34.324609 16839 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfbdddaf01ba43de83d08e7bfd20f033. Latest consensus state: current_term: 1 leader_uuid: "bfbdddaf01ba43de83d08e7bfd20f033" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfbdddaf01ba43de83d08e7bfd20f033" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43479 } } }
I20260812 07:07:34.325071 16838 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfbdddaf01ba43de83d08e7bfd20f033" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfbdddaf01ba43de83d08e7bfd20f033" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43479 } } }
I20260812 07:07:34.325433 16839 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:34.325436 16838 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033 [sys.catalog]: This master's current role is: LEADER
W20260812 07:07:34.329628 16863 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:07:34.329733 16863 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:07:34.329849 16865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:34.331403 16865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:34.338788 16865 catalog_manager.cc:1347] Generated new cluster ID: ed25316ed94846f5b80a591830e155e7
I20260812 07:07:34.338882 16865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:34.360551 16865 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:34.362246 16865 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:34.374487 16865 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfbdddaf01ba43de83d08e7bfd20f033: Generated new TSK 0
I20260812 07:07:34.375435 16865 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:07:34.385927 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 16700
W20260812 07:07:34.386938 16808 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60529
W20260812 07:07:34.387645 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 16700: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:07:34.387809 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 16700
W20260812 07:07:34.411091 16675 connection.cc:535] client connection to 127.0.0.1:43479 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43479): Connection reset by peer (error 104)
2026-08-12T07:07:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2023 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:07:34 dist-test-slave-lgr5 krb5kdc[16877](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:07:34 dist-test-slave-lgr5 krb5kdc[16877](info): set up 2 sockets
Aug 12 07:07:34 dist-test-slave-lgr5 krb5kdc[16877](info): commencing operation
krb5kdc: starting...
W20260812 07:07:35.623840 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.002s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:07:35 dist-test-slave-lgr5 krb5kdc[16877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:07:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:07:35.884980 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34319
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:36.206398 16901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:36.206643 16901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:36.206739 16901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:36.213680 16901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:36.213773 16901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:36.213825 16901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:36.213883 16901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:36.213927 16901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:36.223150 16901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34319
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:36.224251 16901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:36.225411 16901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:36.236351 16912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:36.236404 16915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:36.236436 16913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:36.237478 16901 server_base.cc:1034] running on GCE node
I20260812 07:07:36.238196 16901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:36.239442 16901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:36.240628 16901 hybrid_clock.cc:648] HybridClock initialized: now 1786518456240588 us; error 31 us; skew 500 ppm
I20260812 07:07:36.240940 16901 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:36 dist-test-slave-lgr5 krb5kdc[16877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518456, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:36.243803 16901 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:07:36.245195 16901 webserver.cc:460] Webserver started at http://127.0.0.1:44811/ using document root <none> and password file <none>
I20260812 07:07:36.245787 16901 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:36.245899 16901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:36.246182 16901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:36.248971 16901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "fd1ab6b77c5848a2b2c267022714bdad"
format_stamp: "Formatted at 2026-08-12 07:07:36 on dist-test-slave-lgr5"
server_key: "1b298cded2b8d5871980a7e8737716f0"
server_key_iv: "ee264b94596b9fc20d1abd7752491914"
server_key_version: "encryptionkey@0"
I20260812 07:07:36.249691 16901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "fd1ab6b77c5848a2b2c267022714bdad"
format_stamp: "Formatted at 2026-08-12 07:07:36 on dist-test-slave-lgr5"
server_key: "1b298cded2b8d5871980a7e8737716f0"
server_key_iv: "ee264b94596b9fc20d1abd7752491914"
server_key_version: "encryptionkey@0"
I20260812 07:07:36.254454 16901 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:07:36.257723 16927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:36.258926 16901 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:07:36.259084 16901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "fd1ab6b77c5848a2b2c267022714bdad"
format_stamp: "Formatted at 2026-08-12 07:07:36 on dist-test-slave-lgr5"
server_key: "1b298cded2b8d5871980a7e8737716f0"
server_key_iv: "ee264b94596b9fc20d1abd7752491914"
server_key_version: "encryptionkey@0"
I20260812 07:07:36.259255 16901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:36.264684 16901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:36.268213 16901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:36.268468 16901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:36.285022 16901 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34319
I20260812 07:07:36.285074 17018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34319 every 8 connection(s)
I20260812 07:07:36.286295 16901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:36.287562 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 16901
I20260812 07:07:36.287743 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:07:36.288319 12035 external_mini_cluster.cc:1428] Setting key 3103a6f4f892ffad33aa8dc2595d3cda
I20260812 07:07:36.290485 17019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:36.295439 17030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:36.289957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42911 (local address 127.0.0.1:34319)
0812 07:07:36.290721 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:36.290735 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:36.290775 (+ 40us) server_negotiation.cc:407] Connection header received
0812 07:07:36.292396 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:36.292417 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:36.292728 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:36.293039 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:36.294006 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:36.294013 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:36.294025 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:36.294039 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:36.294123 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:36.294142 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:36.294300 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:36.294396 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 07:07:36.294576 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":319,"threads_started":1}
I20260812 07:07:36.298856 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:07:36.301788 17019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad: Bootstrap starting.
I20260812 07:07:36.304919 17019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:36.304989 17030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:36.302800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49925 (local address 127.0.0.1:34319)
0812 07:07:36.302964 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:36.302981 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:36.303395 (+ 414us) server_negotiation.cc:407] Connection header received
0812 07:07:36.303513 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:36.303518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:36.303569 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:36.303664 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:36.304427 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:36.304432 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:36.304437 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:36.304453 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:36.304495 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:36.304514 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:36.304623 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:36.304669 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 07:07:36.304717 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:07:36.306099 17019 log.cc:824] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:36.308490 17019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad: No bootstrap required, opened a new log
I20260812 07:07:36.315304 17019 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd1ab6b77c5848a2b2c267022714bdad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34319 } }
I20260812 07:07:36.315654 17019 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:36.315765 17019 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd1ab6b77c5848a2b2c267022714bdad, State: Initialized, Role: FOLLOWER
I20260812 07:07:36.316326 17019 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd1ab6b77c5848a2b2c267022714bdad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34319 } }
I20260812 07:07:36.316551 17019 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:36.316687 17019 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:36.316825 17019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:36.318768 17019 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd1ab6b77c5848a2b2c267022714bdad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34319 } }
I20260812 07:07:36.319199 17019 leader_election.cc:302] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd1ab6b77c5848a2b2c267022714bdad; no voters:
I20260812 07:07:36.319696 17019 leader_election.cc:288] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:36.319800 17044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:36.320446 17044 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad [term 1 LEADER]: Becoming Leader. State: Replica: fd1ab6b77c5848a2b2c267022714bdad, State: Running, Role: LEADER
I20260812 07:07:36.320997 17044 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd1ab6b77c5848a2b2c267022714bdad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34319 } }
I20260812 07:07:36.321470 17019 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:36.323484 17048 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd1ab6b77c5848a2b2c267022714bdad. Latest consensus state: current_term: 1 leader_uuid: "fd1ab6b77c5848a2b2c267022714bdad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd1ab6b77c5848a2b2c267022714bdad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34319 } } }
I20260812 07:07:36.323355 17047 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd1ab6b77c5848a2b2c267022714bdad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd1ab6b77c5848a2b2c267022714bdad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34319 } } }
I20260812 07:07:36.323823 17048 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:36.323829 17047 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:36.325291 17053 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:36.330452 17053 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:36.338685 17053 catalog_manager.cc:1347] Generated new cluster ID: 29caf7950af642c0a825caf4fb3dbbd9
I20260812 07:07:36.338773 17053 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:36.350921 17053 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:36.351289 17053 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:36.358285 17053 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd1ab6b77c5848a2b2c267022714bdad: Generated new TSK 0
I20260812 07:07:36.358958 17053 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:07:36.364363 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 16901
W20260812 07:07:36.365337 17010 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42911
W20260812 07:07:36.365943 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 16901: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:07:36.366101 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 16901
2026-08-12T07:07:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1985 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:07:36 dist-test-slave-lgr5 krb5kdc[17081](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:07:36 dist-test-slave-lgr5 krb5kdc[17081](info): set up 2 sockets
Aug 12 07:07:36 dist-test-slave-lgr5 krb5kdc[17081](info): commencing operation
krb5kdc: starting...
W20260812 07:07:37.637283 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:07:37 dist-test-slave-lgr5 krb5kdc[17081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:07:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:07:37.909261 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36335
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36335
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:38.231141 17103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:38.231408 17103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:38.231505 17103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:38.238415 17103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:38.238505 17103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:38.238556 17103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:38.238615 17103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:38.238656 17103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:38.247769 17103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36335
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:38.248847 17103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:38.249970 17103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:38.260952 17114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:38.260979 17111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:38.260959 17110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:38.262084 17103 server_base.cc:1034] running on GCE node
I20260812 07:07:38.262740 17103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:38.263988 17103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:38.265172 17103 hybrid_clock.cc:648] HybridClock initialized: now 1786518458265126 us; error 60 us; skew 500 ppm
I20260812 07:07:38.265488 17103 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:38 dist-test-slave-lgr5 krb5kdc[17081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518458, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:38.268388 17103 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:07:38.269680 17103 webserver.cc:460] Webserver started at http://127.0.0.1:39509/ using document root <none> and password file <none>
I20260812 07:07:38.270300 17103 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:38.270416 17103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:38.270685 17103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:38.273514 17103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "16521e58a40e42638817cd58a0fa9250"
format_stamp: "Formatted at 2026-08-12 07:07:38 on dist-test-slave-lgr5"
server_key: "136cd8ebdde9687626fc2c90ad83913c"
server_key_iv: "7dfdb0b3e92d8d3f570dbe832f90ed0c"
server_key_version: "encryptionkey@0"
I20260812 07:07:38.274251 17103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "16521e58a40e42638817cd58a0fa9250"
format_stamp: "Formatted at 2026-08-12 07:07:38 on dist-test-slave-lgr5"
server_key: "136cd8ebdde9687626fc2c90ad83913c"
server_key_iv: "7dfdb0b3e92d8d3f570dbe832f90ed0c"
server_key_version: "encryptionkey@0"
I20260812 07:07:38.279045 17103 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260812 07:07:38.282248 17122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:38.283455 17103 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 07:07:38.283617 17103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "16521e58a40e42638817cd58a0fa9250"
format_stamp: "Formatted at 2026-08-12 07:07:38 on dist-test-slave-lgr5"
server_key: "136cd8ebdde9687626fc2c90ad83913c"
server_key_iv: "7dfdb0b3e92d8d3f570dbe832f90ed0c"
server_key_version: "encryptionkey@0"
I20260812 07:07:38.283785 17103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:38.297889 17103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:38.301349 17103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:38.301591 17103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:38.318399 17103 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36335
I20260812 07:07:38.318431 17214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36335 every 8 connection(s)
I20260812 07:07:38.319644 17103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:38.321337 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 17103
I20260812 07:07:38.321527 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:07:38.322108 12035 external_mini_cluster.cc:1428] Setting key 3946f2c1f7c3425c0cd606ba87a9bb16
I20260812 07:07:38.323927 17215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 07:07:38 dist-test-slave-lgr5 krb5kdc[17081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 07:07:38.336485 17215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250: Bootstrap starting.
I20260812 07:07:38.339620 17215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:38.340708 17215 log.cc:824] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:38.343681 17215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250: No bootstrap required, opened a new log
I20260812 07:07:38.344403 17221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:38.323757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45753 (local address 127.0.0.1:36335)
0812 07:07:38.324530 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:38.324545 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:38.324592 (+ 47us) server_negotiation.cc:407] Connection header received
0812 07:07:38.326667 (+ 2075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:38.326692 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:38.327192 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:38.327609 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:38.328898 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:38.330861 (+ 1963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:38.332042 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:38.333063 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:38.333255 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:38.337463 (+ 4208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:38.337471 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:38.337491 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:38.337530 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:38.340405 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:38.341228 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:38.341237 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:38.341245 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:38.341494 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:38.341863 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:38.341868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:38.341872 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:38.342363 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:38.342570 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:38.343041 (+ 471us) server_negotiation.cc:299] Negotiation successful
0812 07:07:38.343339 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":329,"threads_started":1}
I20260812 07:07:38.347738 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:07:38.352583 17215 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16521e58a40e42638817cd58a0fa9250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36335 } }
I20260812 07:07:38.352952 17215 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:38.353103 17215 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16521e58a40e42638817cd58a0fa9250, State: Initialized, Role: FOLLOWER
I20260812 07:07:38.353704 17215 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16521e58a40e42638817cd58a0fa9250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36335 } }
I20260812 07:07:38.353983 17215 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:38.354130 17215 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:38.354302 17215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:38.356922 17215 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16521e58a40e42638817cd58a0fa9250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36335 } }
I20260812 07:07:38.357461 17215 leader_election.cc:302] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16521e58a40e42638817cd58a0fa9250; no voters:
I20260812 07:07:38.358059 17215 leader_election.cc:288] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:38.358213 17231 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:38.359335 17231 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250 [term 1 LEADER]: Becoming Leader. State: Replica: 16521e58a40e42638817cd58a0fa9250, State: Running, Role: LEADER
I20260812 07:07:38.360054 17215 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:38.360206 17231 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16521e58a40e42638817cd58a0fa9250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36335 } }
I20260812 07:07:38.363303 17233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16521e58a40e42638817cd58a0fa9250. Latest consensus state: current_term: 1 leader_uuid: "16521e58a40e42638817cd58a0fa9250" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16521e58a40e42638817cd58a0fa9250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36335 } } }
I20260812 07:07:38.364344 17233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:38.363932 17232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16521e58a40e42638817cd58a0fa9250" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16521e58a40e42638817cd58a0fa9250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36335 } } }
I20260812 07:07:38.364852 17232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:38.369611 17221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:38.352847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40293 (local address 127.0.0.1:36335)
0812 07:07:38.353033 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:38.353040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:38.353445 (+ 405us) server_negotiation.cc:407] Connection header received
0812 07:07:38.353580 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:38.353583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:38.353637 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:38.353715 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:38.355501 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:38.356277 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:38.357870 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:38.358589 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:38.358719 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:38.361663 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:38.361673 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:38.361678 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:38.361717 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:38.365548 (+ 3831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:38.366422 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:38.366427 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:38.366431 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:38.366623 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:38.367125 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:38.367130 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:38.367134 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:38.367541 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:38.368809 (+ 1268us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:38.368959 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 07:07:38.369197 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 07:07:38.371706 17242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:38.376801 17242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:38.385525 17242 catalog_manager.cc:1347] Generated new cluster ID: 0d48251f78c1473c9dad826eaa7f66da
I20260812 07:07:38.385623 17242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:38.408190 17242 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:38.409379 17242 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:38.417200 17242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16521e58a40e42638817cd58a0fa9250: Generated new TSK 0
I20260812 07:07:38.417868 17242 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:07:38.428905 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 17103
I20260812 07:07:38.429862 17209 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:38.518878 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 17103
2026-08-12T07:07:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2146 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:07:38 dist-test-slave-lgr5 krb5kdc[17280](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:07:38 dist-test-slave-lgr5 krb5kdc[17280](info): set up 2 sockets
Aug 12 07:07:38 dist-test-slave-lgr5 krb5kdc[17280](info): commencing operation
krb5kdc: starting...
W20260812 07:07:39.813431 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:07:39 dist-test-slave-lgr5 krb5kdc[17280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:07:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:07:40.080127 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41759
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:40.398408 17311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:40.398672 17311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:40.398805 17311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:40.405691 17311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:40.405789 17311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:40.405869 17311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:40.405947 17311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:40.406006 17311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:40.415212 17311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:40.416342 17311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:40.417564 17311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:40.428350 17320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:40.428560 17317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:40.428632 17323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:40.429178 17311 server_base.cc:1034] running on GCE node
I20260812 07:07:40.429924 17311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:40.431175 17311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:40.432366 17311 hybrid_clock.cc:648] HybridClock initialized: now 1786518460432327 us; error 28 us; skew 500 ppm
I20260812 07:07:40.432679 17311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:40 dist-test-slave-lgr5 krb5kdc[17280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518460, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:40.438380 17311 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:07:40.439724 17311 webserver.cc:460] Webserver started at http://127.0.0.1:42079/ using document root <none> and password file <none>
I20260812 07:07:40.440398 17311 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:40.440517 17311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:40.440790 17311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:40.443557 17311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "d5fce93ebd76433798c6099864dfd89d"
format_stamp: "Formatted at 2026-08-12 07:07:40 on dist-test-slave-lgr5"
server_key: "bb0932d12a7d062baf773312cd440128"
server_key_iv: "d7fbd5c66fdf9e580754ac8111c20f40"
server_key_version: "encryptionkey@0"
I20260812 07:07:40.444317 17311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "d5fce93ebd76433798c6099864dfd89d"
format_stamp: "Formatted at 2026-08-12 07:07:40 on dist-test-slave-lgr5"
server_key: "bb0932d12a7d062baf773312cd440128"
server_key_iv: "d7fbd5c66fdf9e580754ac8111c20f40"
server_key_version: "encryptionkey@0"
I20260812 07:07:40.449117 17311 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:07:40.452229 17331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:40.453398 17311 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 07:07:40.453557 17311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "d5fce93ebd76433798c6099864dfd89d"
format_stamp: "Formatted at 2026-08-12 07:07:40 on dist-test-slave-lgr5"
server_key: "bb0932d12a7d062baf773312cd440128"
server_key_iv: "d7fbd5c66fdf9e580754ac8111c20f40"
server_key_version: "encryptionkey@0"
I20260812 07:07:40.453718 17311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:40.471958 17311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:40.475472 17311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:40.475704 17311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:40.492682 17311 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41759
I20260812 07:07:40.492724 17426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41759 every 8 connection(s)
I20260812 07:07:40.493911 17311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:40.497336 17427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:40.502066 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 17311
I20260812 07:07:40.502259 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:07:40.502807 12035 external_mini_cluster.cc:1428] Setting key 912318fb00572c01855d1938e76e2b02
I20260812 07:07:40.508129 17427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d: Bootstrap starting.
I20260812 07:07:40.510964 17427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:40.511934 17427 log.cc:824] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:40.514907 17427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d: No bootstrap required, opened a new log
I20260812 07:07:40.514978 17433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:40.504568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37099 (local address 127.0.0.1:41759)
0812 07:07:40.505340 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:40.505351 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:40.505391 (+ 40us) server_negotiation.cc:407] Connection header received
0812 07:07:40.507082 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:40.507103 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:40.507435 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:40.507766 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:40.508290 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:40.509549 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:40.510808 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:40.511502 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:40.511661 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:40.512754 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:40.512767 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:40.512785 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:40.513471 (+ 686us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:40.513551 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:40.513576 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:40.513674 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:40.513900 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 07:07:40.514140 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":372,"threads_started":1}
I20260812 07:07:40.518386 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:07:40.522899 17427 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5fce93ebd76433798c6099864dfd89d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41759 } }
I20260812 07:07:40.523247 17427 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:40.523362 17427 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5fce93ebd76433798c6099864dfd89d, State: Initialized, Role: FOLLOWER
I20260812 07:07:40.523949 17427 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5fce93ebd76433798c6099864dfd89d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41759 } }
I20260812 07:07:40.524220 17427 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:40.524346 17427 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:40.524509 17427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:40.527267 17427 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5fce93ebd76433798c6099864dfd89d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41759 } }
I20260812 07:07:40.527791 17427 leader_election.cc:302] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5fce93ebd76433798c6099864dfd89d; no voters:
I20260812 07:07:40.528429 17427 leader_election.cc:288] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:40.528517 17442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:40.528870 17433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:40.522380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41699 (local address 127.0.0.1:41759)
0812 07:07:40.522511 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:40.522514 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:40.522939 (+ 425us) server_negotiation.cc:407] Connection header received
0812 07:07:40.523112 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:40.523116 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:40.523168 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:40.523247 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:40.523827 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:40.524687 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:40.526480 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:40.526888 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:40.526972 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:40.527980 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:40.527986 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:40.527991 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:40.528009 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:40.528047 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:40.528062 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:40.528165 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:40.528332 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 07:07:40.528432 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 07:07:40.530524 17427 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:40.532370 17442 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d [term 1 LEADER]: Becoming Leader. State: Replica: d5fce93ebd76433798c6099864dfd89d, State: Running, Role: LEADER
I20260812 07:07:40.533007 17442 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5fce93ebd76433798c6099864dfd89d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41759 } }
I20260812 07:07:40.536196 17444 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5fce93ebd76433798c6099864dfd89d. Latest consensus state: current_term: 1 leader_uuid: "d5fce93ebd76433798c6099864dfd89d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5fce93ebd76433798c6099864dfd89d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41759 } } }
I20260812 07:07:40.536415 17444 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:40.535833 17443 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5fce93ebd76433798c6099864dfd89d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5fce93ebd76433798c6099864dfd89d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41759 } } }
I20260812 07:07:40.536755 17443 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:40.537781 17466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:40.541841 17466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:40.549547 17466 catalog_manager.cc:1347] Generated new cluster ID: 1015ef2306e84df395752f0b5e04cd4c
I20260812 07:07:40.549641 17466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:40.561903 17466 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:40.563627 17466 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:40.575448 17466 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5fce93ebd76433798c6099864dfd89d: Generated new TSK 0
I20260812 07:07:40.576399 17466 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:07:40.585875 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 17311
W20260812 07:07:40.586844 17420 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37099
W20260812 07:07:40.587462 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 17311: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:07:40.587587 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 17311
2026-08-12T07:07:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2071 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:07:40 dist-test-slave-lgr5 krb5kdc[17481](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:07:40 dist-test-slave-lgr5 krb5kdc[17481](info): set up 2 sockets
Aug 12 07:07:40 dist-test-slave-lgr5 krb5kdc[17481](info): commencing operation
krb5kdc: starting...
W20260812 07:07:41.844841 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.004s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:07:42 dist-test-slave-lgr5 krb5kdc[17481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:07:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:07:42.119060 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45047
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45047
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:42.439292 17503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:42.439574 17503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:42.439704 17503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:42.446676 17503 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:42.446774 17503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:42.446851 17503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:42.446929 17503 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:42.446993 17503 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:42.456306 17503 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:42.457392 17503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:42.458583 17503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:42.469427 17509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:42.469460 17510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:42.469797 17513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:42.470504 17503 server_base.cc:1034] running on GCE node
I20260812 07:07:42.471144 17503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:42.472404 17503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:42.473640 17503 hybrid_clock.cc:648] HybridClock initialized: now 1786518462473584 us; error 41 us; skew 500 ppm
I20260812 07:07:42.473999 17503 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:42 dist-test-slave-lgr5 krb5kdc[17481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518462, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:42.477010 17503 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:07:42.478322 17503 webserver.cc:460] Webserver started at http://127.0.0.1:40439/ using document root <none> and password file <none>
I20260812 07:07:42.478935 17503 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:42.479038 17503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:42.479328 17503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:42.482225 17503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "eb68b73503b2464e89db0d91e2b132a4"
format_stamp: "Formatted at 2026-08-12 07:07:42 on dist-test-slave-lgr5"
server_key: "80f4a27531a854914943c4976f436781"
server_key_iv: "42eebd8c0c2b9568d5bf41d5862867a5"
server_key_version: "encryptionkey@0"
I20260812 07:07:42.482944 17503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "eb68b73503b2464e89db0d91e2b132a4"
format_stamp: "Formatted at 2026-08-12 07:07:42 on dist-test-slave-lgr5"
server_key: "80f4a27531a854914943c4976f436781"
server_key_iv: "42eebd8c0c2b9568d5bf41d5862867a5"
server_key_version: "encryptionkey@0"
I20260812 07:07:42.487776 17503 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 07:07:42.491043 17528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:42.492281 17503 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:07:42.492429 17503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "eb68b73503b2464e89db0d91e2b132a4"
format_stamp: "Formatted at 2026-08-12 07:07:42 on dist-test-slave-lgr5"
server_key: "80f4a27531a854914943c4976f436781"
server_key_iv: "42eebd8c0c2b9568d5bf41d5862867a5"
server_key_version: "encryptionkey@0"
I20260812 07:07:42.492604 17503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:42.498188 17503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:42.502131 17503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:42.502393 17503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:42.519093 17503 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45047
I20260812 07:07:42.519137 17615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45047 every 8 connection(s)
I20260812 07:07:42.520401 17503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:42.523693 17619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:42.530066 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 17503
I20260812 07:07:42.530224 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:07:42.530658 12035 external_mini_cluster.cc:1428] Setting key aade885f1b827ebb6369eebd45694dab
I20260812 07:07:42.533399 17619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4: Bootstrap starting.
I20260812 07:07:42.536535 17619 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:42.537596 17619 log.cc:824] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:42.537722 17626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:42.532195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54419 (local address 127.0.0.1:45047)
0812 07:07:42.532962 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:42.532973 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:42.533013 (+ 40us) server_negotiation.cc:407] Connection header received
0812 07:07:42.534655 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:42.534680 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:42.534970 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:42.535277 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:42.536280 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:42.536286 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:42.536298 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:42.536312 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:42.536365 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:42.536384 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:42.536538 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:42.536705 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 07:07:42.536873 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":346,"threads_started":1}
I20260812 07:07:42.540256 17619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4: No bootstrap required, opened a new log
I20260812 07:07:42.541023 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:07:42.548130 17626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:42.545978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42643 (local address 127.0.0.1:45047)
0812 07:07:42.546156 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:42.546162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:42.546554 (+ 392us) server_negotiation.cc:407] Connection header received
0812 07:07:42.546662 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:42.546670 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:42.546721 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:42.546843 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:42.547546 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:42.547552 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:42.547556 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:42.547578 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:42.547624 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:42.547642 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:42.547755 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:42.547798 (+ 43us) server_negotiation.cc:299] Negotiation successful
0812 07:07:42.547855 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:07:42.549201 17619 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb68b73503b2464e89db0d91e2b132a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45047 } }
I20260812 07:07:42.549499 17619 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:42.549567 17619 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb68b73503b2464e89db0d91e2b132a4, State: Initialized, Role: FOLLOWER
I20260812 07:07:42.550031 17619 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb68b73503b2464e89db0d91e2b132a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45047 } }
I20260812 07:07:42.550236 17619 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:42.550326 17619 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:42.550479 17619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:42.552474 17619 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb68b73503b2464e89db0d91e2b132a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45047 } }
I20260812 07:07:42.552879 17619 leader_election.cc:302] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb68b73503b2464e89db0d91e2b132a4; no voters:
I20260812 07:07:42.553318 17619 leader_election.cc:288] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:42.553432 17639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:42.554044 17639 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4 [term 1 LEADER]: Becoming Leader. State: Replica: eb68b73503b2464e89db0d91e2b132a4, State: Running, Role: LEADER
I20260812 07:07:42.554535 17639 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb68b73503b2464e89db0d91e2b132a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45047 } }
I20260812 07:07:42.555231 17619 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:42.557175 17640 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb68b73503b2464e89db0d91e2b132a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb68b73503b2464e89db0d91e2b132a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45047 } } }
I20260812 07:07:42.557483 17640 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:42.559037 17646 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb68b73503b2464e89db0d91e2b132a4. Latest consensus state: current_term: 1 leader_uuid: "eb68b73503b2464e89db0d91e2b132a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb68b73503b2464e89db0d91e2b132a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45047 } } }
I20260812 07:07:42.559279 17646 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:42.561156 17651 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:42.565861 17651 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:42.574599 17651 catalog_manager.cc:1347] Generated new cluster ID: de28ec31c16d4072a822682f01b4114c
I20260812 07:07:42.574702 17651 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:42.590961 17651 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:42.591427 17651 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:42.607210 17651 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb68b73503b2464e89db0d91e2b132a4: Generated new TSK 0
I20260812 07:07:42.607901 17651 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:07:42.614848 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 17503
W20260812 07:07:42.616183 17609 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54419
W20260812 07:07:42.616799 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 17503: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:07:42.616946 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 17503
2026-08-12T07:07:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2031 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:07:42 dist-test-slave-lgr5 krb5kdc[17674](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:07:42 dist-test-slave-lgr5 krb5kdc[17674](info): set up 2 sockets
Aug 12 07:07:42 dist-test-slave-lgr5 krb5kdc[17674](info): commencing operation
krb5kdc: starting...
W20260812 07:07:43.884202 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:07:44 dist-test-slave-lgr5 krb5kdc[17674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518464, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:07:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:07:44.162828 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37191
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:44.495775 17693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:44.496047 17693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:44.496201 17693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:44.503264 17693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:44.503392 17693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:44.503459 17693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:44.503530 17693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:44.503580 17693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:44.513391 17693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:44.514523 17693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:44.515802 17693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:44.527341 17704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:44.527426 17700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:44.527400 17701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:44.528357 17693 server_base.cc:1034] running on GCE node
I20260812 07:07:44.529024 17693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:44.530284 17693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:44.531507 17693 hybrid_clock.cc:648] HybridClock initialized: now 1786518464531448 us; error 41 us; skew 500 ppm
I20260812 07:07:44.531852 17693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:44 dist-test-slave-lgr5 krb5kdc[17674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518464, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:44.535003 17693 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:07:44.536510 17693 webserver.cc:460] Webserver started at http://127.0.0.1:42839/ using document root <none> and password file <none>
I20260812 07:07:44.537154 17693 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:44.537290 17693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:44.537600 17693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:44.540609 17693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "bf29e6d03c474336b8cba55cfd2491eb"
format_stamp: "Formatted at 2026-08-12 07:07:44 on dist-test-slave-lgr5"
server_key: "bb7aac8ab15419800af48576f193ca0e"
server_key_iv: "0dbd349e23f07c0f8e54050768f13d2a"
server_key_version: "encryptionkey@0"
I20260812 07:07:44.541421 17693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "bf29e6d03c474336b8cba55cfd2491eb"
format_stamp: "Formatted at 2026-08-12 07:07:44 on dist-test-slave-lgr5"
server_key: "bb7aac8ab15419800af48576f193ca0e"
server_key_iv: "0dbd349e23f07c0f8e54050768f13d2a"
server_key_version: "encryptionkey@0"
I20260812 07:07:44.546459 17693 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:07:44.549856 17719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:44.551061 17693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:07:44.551239 17693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "bf29e6d03c474336b8cba55cfd2491eb"
format_stamp: "Formatted at 2026-08-12 07:07:44 on dist-test-slave-lgr5"
server_key: "bb7aac8ab15419800af48576f193ca0e"
server_key_iv: "0dbd349e23f07c0f8e54050768f13d2a"
server_key_version: "encryptionkey@0"
I20260812 07:07:44.551438 17693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:44.568883 17693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:44.573091 17693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:44.573372 17693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:44.592265 17693 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37191
I20260812 07:07:44.592283 17822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37191 every 8 connection(s)
I20260812 07:07:44.593693 17693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:44.597239 17824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:44.601329 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 17693
I20260812 07:07:44.601521 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:07:44.601975 12035 external_mini_cluster.cc:1428] Setting key 915086a09b7e33aa20deaf5cdbb9e024
I20260812 07:07:44.607640 17824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb: Bootstrap starting.
I20260812 07:07:44.611058 17824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:44.612289 17824 log.cc:824] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:44.615538 17824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb: No bootstrap required, opened a new log
Aug 12 07:07:44 dist-test-slave-lgr5 krb5kdc[17674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518464, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 07:07:44.623034 17824 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf29e6d03c474336b8cba55cfd2491eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37191 } }
I20260812 07:07:44.623371 17824 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:44.623474 17824 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf29e6d03c474336b8cba55cfd2491eb, State: Initialized, Role: FOLLOWER
I20260812 07:07:44.624054 17824 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf29e6d03c474336b8cba55cfd2491eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37191 } }
I20260812 07:07:44.624327 17824 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:44.624519 17824 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:44.624728 17824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:44.625790 17827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:44.603793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:55205 (local address 127.0.0.1:37191)
0812 07:07:44.604642 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:44.604654 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:44.604706 (+ 52us) server_negotiation.cc:407] Connection header received
0812 07:07:44.606562 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:44.606585 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:44.606961 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:44.607290 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:44.609130 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:44.610417 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:44.612173 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:44.613119 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:44.613332 (+ 213us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:44.617946 (+ 4614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:44.617954 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:44.617967 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:44.618008 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:44.621287 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:44.622150 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:44.622157 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:44.622167 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:44.622364 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:44.622852 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:44.622856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:44.622858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:44.623434 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:44.623758 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:44.624392 (+ 634us) server_negotiation.cc:299] Negotiation successful
0812 07:07:44.624742 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":375,"threads_started":1}
I20260812 07:07:44.627027 17824 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf29e6d03c474336b8cba55cfd2491eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37191 } }
I20260812 07:07:44.627485 17824 leader_election.cc:302] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf29e6d03c474336b8cba55cfd2491eb; no voters:
I20260812 07:07:44.628072 17824 leader_election.cc:288] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:44.628211 17830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:44.629269 17830 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb [term 1 LEADER]: Becoming Leader. State: Replica: bf29e6d03c474336b8cba55cfd2491eb, State: Running, Role: LEADER
I20260812 07:07:44.629871 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:07:44.629889 17830 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf29e6d03c474336b8cba55cfd2491eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37191 } }
I20260812 07:07:44.630195 17824 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:44.632514 17832 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf29e6d03c474336b8cba55cfd2491eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf29e6d03c474336b8cba55cfd2491eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37191 } } }
I20260812 07:07:44.632896 17832 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:44.637749 17835 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf29e6d03c474336b8cba55cfd2491eb. Latest consensus state: current_term: 1 leader_uuid: "bf29e6d03c474336b8cba55cfd2491eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf29e6d03c474336b8cba55cfd2491eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37191 } } }
I20260812 07:07:44.638088 17835 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:44.640825 17846 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:44.646188 17846 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:44.666968 17846 catalog_manager.cc:1347] Generated new cluster ID: beb89ed3c0f84ffa918e0dd0cdb845a6
I20260812 07:07:44.667084 17846 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:44.682437 17827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:44.652651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:36295 (local address 127.0.0.1:37191)
0812 07:07:44.652809 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:44.652814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:44.666188 (+ 13374us) server_negotiation.cc:407] Connection header received
0812 07:07:44.666365 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:44.666370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:44.666454 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:44.666562 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:44.668617 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:44.669729 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:44.671443 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:44.672125 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:44.672213 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:44.675622 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:44.675629 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:44.675634 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:44.675671 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:44.678959 (+ 3288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:44.679827 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:44.679833 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:44.679837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:44.679981 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:44.680615 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:44.680621 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:44.680624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:44.681061 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:44.681304 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:44.681914 (+ 610us) server_negotiation.cc:299] Negotiation successful
0812 07:07:44.682081 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:07:44.688697 17846 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:44.690449 17846 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:44.704016 17846 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf29e6d03c474336b8cba55cfd2491eb: Generated new TSK 0
I20260812 07:07:44.705013 17846 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:07:44.714721 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 17693
I20260812 07:07:44.715888 17815 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:44.807014 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 17693
W20260812 07:07:44.830999 17669 connection.cc:535] client connection to 127.0.0.1:37191 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37191): Connection reset by peer (error 104)
2026-08-12T07:07:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2185 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:07:44 dist-test-slave-lgr5 krb5kdc[17881](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:07:44 dist-test-slave-lgr5 krb5kdc[17881](info): set up 2 sockets
Aug 12 07:07:44 dist-test-slave-lgr5 krb5kdc[17881](info): commencing operation
krb5kdc: starting...
W20260812 07:07:46.068516 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.003s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:07:46 dist-test-slave-lgr5 krb5kdc[17881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:07:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:07:46.347574 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44343
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:46.670449 17912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:46.670709 17912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:46.670847 17912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:46.677757 17912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:46.677884 17912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:46.677965 17912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:46.678050 17912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:46.678112 17912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:46.687431 17912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:46.688675 17912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:46.690033 17912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:46.701599 17921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:46.701555 17926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:46.701632 17920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:46.702373 17912 server_base.cc:1034] running on GCE node
I20260812 07:07:46.703092 17912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:46.704478 17912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:46.705680 17912 hybrid_clock.cc:648] HybridClock initialized: now 1786518466705634 us; error 37 us; skew 500 ppm
I20260812 07:07:46.706023 17912 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:46 dist-test-slave-lgr5 krb5kdc[17881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518466, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:46.711727 17912 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:07:46.713138 17912 webserver.cc:460] Webserver started at http://127.0.0.1:40385/ using document root <none> and password file <none>
I20260812 07:07:46.713766 17912 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:46.713898 17912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:46.714190 17912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:46.717128 17912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "62bac35d305045e4ad5895481a5df421"
format_stamp: "Formatted at 2026-08-12 07:07:46 on dist-test-slave-lgr5"
server_key: "375a70b15fbba175e635ddfcb06fa323"
server_key_iv: "ec20ba5cccaaa5c201433c6b05594f2b"
server_key_version: "encryptionkey@0"
I20260812 07:07:46.717882 17912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "62bac35d305045e4ad5895481a5df421"
format_stamp: "Formatted at 2026-08-12 07:07:46 on dist-test-slave-lgr5"
server_key: "375a70b15fbba175e635ddfcb06fa323"
server_key_iv: "ec20ba5cccaaa5c201433c6b05594f2b"
server_key_version: "encryptionkey@0"
I20260812 07:07:46.722779 17912 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 07:07:46.726025 17940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:46.727200 17912 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:07:46.727349 17912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "62bac35d305045e4ad5895481a5df421"
format_stamp: "Formatted at 2026-08-12 07:07:46 on dist-test-slave-lgr5"
server_key: "375a70b15fbba175e635ddfcb06fa323"
server_key_iv: "ec20ba5cccaaa5c201433c6b05594f2b"
server_key_version: "encryptionkey@0"
I20260812 07:07:46.727515 17912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:46.741077 17912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:46.744948 17912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:46.745196 17912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:46.761996 17912 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44343
I20260812 07:07:46.762030 18046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44343 every 8 connection(s)
I20260812 07:07:46.763182 17912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:46.766958 18047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:46.770882 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 17912
I20260812 07:07:46.771040 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:07:46.771456 12035 external_mini_cluster.cc:1428] Setting key 1d705a9b75918b5fcc1ff7d69a458909
I20260812 07:07:46.779131 18047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421: Bootstrap starting.
I20260812 07:07:46.782454 18047 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:46.783571 18047 log.cc:824] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:46.783953 18051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:46.773084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:57777 (local address 127.0.0.1:44343)
0812 07:07:46.773835 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:46.773846 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:46.773883 (+ 37us) server_negotiation.cc:407] Connection header received
0812 07:07:46.775423 (+ 1540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:46.775445 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:46.775829 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:46.776160 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:46.776714 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:46.778031 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:46.779794 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:46.780487 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:46.780624 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:46.781809 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:46.781817 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:46.781829 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:46.782300 (+ 471us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:46.782379 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:46.782405 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:46.782547 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:46.782831 (+ 284us) server_negotiation.cc:299] Negotiation successful
0812 07:07:46.783017 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":328,"threads_started":1}
I20260812 07:07:46.786482 18047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421: No bootstrap required, opened a new log
I20260812 07:07:46.787842 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:07:46.795442 18047 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62bac35d305045e4ad5895481a5df421" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44343 } }
I20260812 07:07:46.795760 18047 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:46.795852 18047 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62bac35d305045e4ad5895481a5df421, State: Initialized, Role: FOLLOWER
I20260812 07:07:46.796389 18047 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62bac35d305045e4ad5895481a5df421" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44343 } }
I20260812 07:07:46.796607 18047 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:46.796747 18047 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:46.796913 18047 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:46.799027 18051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:46.792972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:39591 (local address 127.0.0.1:44343)
0812 07:07:46.793180 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:46.793187 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:46.793683 (+ 496us) server_negotiation.cc:407] Connection header received
0812 07:07:46.793842 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:46.793846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:46.793904 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:46.793999 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:46.794487 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:46.795233 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:46.796697 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:46.797172 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:46.797278 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:46.798348 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:46.798354 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:46.798358 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:46.798373 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:46.798424 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:46.798439 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:46.798527 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:46.798722 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 07:07:46.798770 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:07:46.799090 18047 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62bac35d305045e4ad5895481a5df421" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44343 } }
I20260812 07:07:46.799685 18047 leader_election.cc:302] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62bac35d305045e4ad5895481a5df421; no voters:
I20260812 07:07:46.800305 18047 leader_election.cc:288] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:46.800418 18067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:46.801106 18067 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421 [term 1 LEADER]: Becoming Leader. State: Replica: 62bac35d305045e4ad5895481a5df421, State: Running, Role: LEADER
I20260812 07:07:46.801757 18067 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62bac35d305045e4ad5895481a5df421" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44343 } }
I20260812 07:07:46.802177 18047 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:46.804349 18069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62bac35d305045e4ad5895481a5df421. Latest consensus state: current_term: 1 leader_uuid: "62bac35d305045e4ad5895481a5df421" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62bac35d305045e4ad5895481a5df421" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44343 } } }
I20260812 07:07:46.804777 18069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:46.804602 18068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62bac35d305045e4ad5895481a5df421" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62bac35d305045e4ad5895481a5df421" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44343 } } }
I20260812 07:07:46.804899 18068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:46.807343 18076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:46.812667 18076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:46.821472 18076 catalog_manager.cc:1347] Generated new cluster ID: 8daf48f9d28e494bb735227a78f21573
I20260812 07:07:46.821580 18076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:46.836186 18076 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:46.837299 18076 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:46.843782 18076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62bac35d305045e4ad5895481a5df421: Generated new TSK 0
I20260812 07:07:46.844481 18076 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:07:46.850379 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 17912
W20260812 07:07:46.851554 18036 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.158:57777
W20260812 07:07:46.852355 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 17912: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:07:46.852487 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 17912
W20260812 07:07:46.877650 17879 connection.cc:535] client connection to 127.0.0.1:44343 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44343): Connection reset by peer (error 104)
2026-08-12T07:07:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2046 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:07:46 dist-test-slave-lgr5 krb5kdc[18112](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:07:46 dist-test-slave-lgr5 krb5kdc[18112](info): set up 2 sockets
Aug 12 07:07:46 dist-test-slave-lgr5 krb5kdc[18112](info): commencing operation
krb5kdc: starting...
W20260812 07:07:48.116500 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:07:48 dist-test-slave-lgr5 krb5kdc[18112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:07:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:07:48.380633 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34215
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:48.712072 18138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:48.712385 18138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:48.712536 18138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:48.719766 18138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:48.719879 18138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:48.719969 18138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:48.720064 18138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:48.720145 18138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:48.729697 18138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:48.730855 18138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:48.732152 18138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:48.743746 18144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:48.743788 18150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:48.743695 18146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:48.744596 18138 server_base.cc:1034] running on GCE node
I20260812 07:07:48.745280 18138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:48.746511 18138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:48.747714 18138 hybrid_clock.cc:648] HybridClock initialized: now 1786518468747665 us; error 45 us; skew 500 ppm
I20260812 07:07:48.748028 18138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:48 dist-test-slave-lgr5 krb5kdc[18112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518468, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:48.751015 18138 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:07:48.752514 18138 webserver.cc:460] Webserver started at http://127.0.0.1:42771/ using document root <none> and password file <none>
I20260812 07:07:48.753144 18138 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:48.753273 18138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:48.753588 18138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:48.756590 18138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "4c1601d13ffc4092888e1987acd96602"
format_stamp: "Formatted at 2026-08-12 07:07:48 on dist-test-slave-lgr5"
server_key: "c1e60644abf3c311face51b49c322a3a"
server_key_iv: "0298e82c7269645ff95432335d04c432"
server_key_version: "encryptionkey@0"
I20260812 07:07:48.757388 18138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "4c1601d13ffc4092888e1987acd96602"
format_stamp: "Formatted at 2026-08-12 07:07:48 on dist-test-slave-lgr5"
server_key: "c1e60644abf3c311face51b49c322a3a"
server_key_iv: "0298e82c7269645ff95432335d04c432"
server_key_version: "encryptionkey@0"
I20260812 07:07:48.762413 18138 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260812 07:07:48.765779 18161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:48.766997 18138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:07:48.767158 18138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "4c1601d13ffc4092888e1987acd96602"
format_stamp: "Formatted at 2026-08-12 07:07:48 on dist-test-slave-lgr5"
server_key: "c1e60644abf3c311face51b49c322a3a"
server_key_iv: "0298e82c7269645ff95432335d04c432"
server_key_version: "encryptionkey@0"
I20260812 07:07:48.767344 18138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:48.773200 18138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:48.778414 18138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:48.778738 18138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:48.797173 18138 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34215
I20260812 07:07:48.797211 18256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34215 every 8 connection(s)
I20260812 07:07:48.798537 18138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:48.801957 18261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:48.803032 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 18138
I20260812 07:07:48.803190 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:07:48.803663 12035 external_mini_cluster.cc:1428] Setting key ebcc2c6e81d9e93bd0e47b9eb6180010
I20260812 07:07:48.811594 18265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:48.805452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:58833 (local address 127.0.0.1:34215)
0812 07:07:48.806282 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:48.806297 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:48.806348 (+ 51us) server_negotiation.cc:407] Connection header received
0812 07:07:48.808153 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:48.808182 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:48.808580 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:48.808998 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:48.809863 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:48.809871 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:48.809886 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:48.809908 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:48.809967 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:48.810002 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:48.810192 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:48.810313 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 07:07:48.810486 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":319,"threads_started":1}
I20260812 07:07:48.815346 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:07:48.815810 18261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602: Bootstrap starting.
I20260812 07:07:48.819396 18261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:48.820675 18261 log.cc:824] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:48.823099 18265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:48.820621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:35499 (local address 127.0.0.1:34215)
0812 07:07:48.820784 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:48.820789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:48.821232 (+ 443us) server_negotiation.cc:407] Connection header received
0812 07:07:48.821407 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:48.821411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:48.821457 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:48.821560 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:48.822452 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:48.822459 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:48.822464 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:48.822483 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:48.822520 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:48.822537 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:48.822638 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:48.822758 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 07:07:48.822826 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:07:48.823153 18261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602: No bootstrap required, opened a new log
I20260812 07:07:48.830276 18261 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1601d13ffc4092888e1987acd96602" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34215 } }
I20260812 07:07:48.830585 18261 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:48.830683 18261 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c1601d13ffc4092888e1987acd96602, State: Initialized, Role: FOLLOWER
I20260812 07:07:48.831207 18261 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1601d13ffc4092888e1987acd96602" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34215 } }
I20260812 07:07:48.831418 18261 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:48.831550 18261 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:48.831688 18261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:48.833730 18261 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1601d13ffc4092888e1987acd96602" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34215 } }
I20260812 07:07:48.834167 18261 leader_election.cc:302] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c1601d13ffc4092888e1987acd96602; no voters:
I20260812 07:07:48.834669 18261 leader_election.cc:288] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:48.834829 18277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:48.835834 18277 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602 [term 1 LEADER]: Becoming Leader. State: Replica: 4c1601d13ffc4092888e1987acd96602, State: Running, Role: LEADER
I20260812 07:07:48.836414 18277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1601d13ffc4092888e1987acd96602" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34215 } }
I20260812 07:07:48.836669 18261 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:48.839094 18280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c1601d13ffc4092888e1987acd96602" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1601d13ffc4092888e1987acd96602" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34215 } } }
I20260812 07:07:48.839161 18283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c1601d13ffc4092888e1987acd96602. Latest consensus state: current_term: 1 leader_uuid: "4c1601d13ffc4092888e1987acd96602" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1601d13ffc4092888e1987acd96602" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34215 } } }
I20260812 07:07:48.839557 18280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:48.839557 18283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:48.841869 18288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:48.847553 18288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:48.855492 18288 catalog_manager.cc:1347] Generated new cluster ID: d3429fd535734849bd4c5d769a51c7bf
I20260812 07:07:48.855580 18288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:48.863214 18288 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:48.863673 18288 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:48.871929 18288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c1601d13ffc4092888e1987acd96602: Generated new TSK 0
I20260812 07:07:48.872777 18288 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:07:48.887672 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 18138
W20260812 07:07:48.888767 18249 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.158:58833
W20260812 07:07:48.889458 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 18138: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:07:48.889591 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 18138
2026-08-12T07:07:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2037 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:07:49 dist-test-slave-lgr5 krb5kdc[18313](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:07:49 dist-test-slave-lgr5 krb5kdc[18313](info): set up 2 sockets
Aug 12 07:07:49 dist-test-slave-lgr5 krb5kdc[18313](info): commencing operation
krb5kdc: starting...
W20260812 07:07:50.145318 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:07:50 dist-test-slave-lgr5 krb5kdc[18313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:07:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:07:50.412014 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34193
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:50.740593 18334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:50.740885 18334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:50.741037 18334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:50.748039 18334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:50.748180 18334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:50.748270 18334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:50.748374 18334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:50.748445 18334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:50.757974 18334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34193
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:50.759130 18334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:50.760416 18334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:50.771656 18344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:50.771623 18347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:50.771689 18343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:50.772699 18334 server_base.cc:1034] running on GCE node
I20260812 07:07:50.773468 18334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:50.774755 18334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:50.775961 18334 hybrid_clock.cc:648] HybridClock initialized: now 1786518470775907 us; error 40 us; skew 500 ppm
I20260812 07:07:50.776331 18334 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:50 dist-test-slave-lgr5 krb5kdc[18313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518470, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:50.779367 18334 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:07:50.780890 18334 webserver.cc:460] Webserver started at http://127.0.0.1:33197/ using document root <none> and password file <none>
I20260812 07:07:50.781662 18334 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:50.781811 18334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:50.782191 18334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:50.785655 18334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "40aff7a8909440479bf926695e4b679d"
format_stamp: "Formatted at 2026-08-12 07:07:50 on dist-test-slave-lgr5"
server_key: "602768cb5ad6fff034c611d94c1820d6"
server_key_iv: "dc925d465595616845eb730496716d80"
server_key_version: "encryptionkey@0"
I20260812 07:07:50.786473 18334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "40aff7a8909440479bf926695e4b679d"
format_stamp: "Formatted at 2026-08-12 07:07:50 on dist-test-slave-lgr5"
server_key: "602768cb5ad6fff034c611d94c1820d6"
server_key_iv: "dc925d465595616845eb730496716d80"
server_key_version: "encryptionkey@0"
I20260812 07:07:50.791622 18334 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 07:07:50.795135 18363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:50.796424 18334 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:07:50.796602 18334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "40aff7a8909440479bf926695e4b679d"
format_stamp: "Formatted at 2026-08-12 07:07:50 on dist-test-slave-lgr5"
server_key: "602768cb5ad6fff034c611d94c1820d6"
server_key_iv: "dc925d465595616845eb730496716d80"
server_key_version: "encryptionkey@0"
I20260812 07:07:50.796784 18334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:50.832741 18334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:50.836588 18334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:50.836860 18334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:50.854835 18334 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34193
I20260812 07:07:50.854875 18451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34193 every 8 connection(s)
I20260812 07:07:50.856249 18334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:50.859758 18453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:50.864455 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 18334
I20260812 07:07:50.864643 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:07:50.865203 12035 external_mini_cluster.cc:1428] Setting key 4a0d42e170fcd5da1eec3bf366320afc
I20260812 07:07:50.870658 18453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d: Bootstrap starting.
I20260812 07:07:50.874109 18453 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:50.875281 18453 log.cc:824] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:50.878398 18453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d: No bootstrap required, opened a new log
Aug 12 07:07:50 dist-test-slave-lgr5 krb5kdc[18313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 07:07:50.886034 18453 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40aff7a8909440479bf926695e4b679d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34193 } }
I20260812 07:07:50.886420 18453 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:50.886526 18453 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40aff7a8909440479bf926695e4b679d, State: Initialized, Role: FOLLOWER
I20260812 07:07:50.887053 18453 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40aff7a8909440479bf926695e4b679d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34193 } }
I20260812 07:07:50.887272 18453 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:50.887449 18459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:50.866910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:33485 (local address 127.0.0.1:34193)
0812 07:07:50.867649 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:50.867660 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:50.867702 (+ 42us) server_negotiation.cc:407] Connection header received
0812 07:07:50.869423 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:50.869445 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:50.869794 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:50.870089 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:50.871859 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:50.873205 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:50.874905 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:50.875591 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:50.875762 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:50.880285 (+ 4523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:50.880296 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:50.880309 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:50.880347 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:50.883287 (+ 2940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:50.884015 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:50.884021 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:50.884027 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:50.884227 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:50.884675 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:50.884679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:50.884681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:50.885221 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:50.885517 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:50.886024 (+ 507us) server_negotiation.cc:299] Negotiation successful
0812 07:07:50.886320 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":332,"threads_started":1}
I20260812 07:07:50.887414 18453 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:50.887648 18453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:50.890410 18453 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40aff7a8909440479bf926695e4b679d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34193 } }
I20260812 07:07:50.890928 18453 leader_election.cc:302] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40aff7a8909440479bf926695e4b679d; no voters:
I20260812 07:07:50.891212 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:07:50.891512 18453 leader_election.cc:288] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:50.891624 18461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:50.893246 18453 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:50.900825 18461 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d [term 1 LEADER]: Becoming Leader. State: Replica: 40aff7a8909440479bf926695e4b679d, State: Running, Role: LEADER
I20260812 07:07:50.901623 18461 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40aff7a8909440479bf926695e4b679d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34193 } }
I20260812 07:07:50.909930 18462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40aff7a8909440479bf926695e4b679d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40aff7a8909440479bf926695e4b679d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34193 } } }
I20260812 07:07:50.910009 18463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40aff7a8909440479bf926695e4b679d. Latest consensus state: current_term: 1 leader_uuid: "40aff7a8909440479bf926695e4b679d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40aff7a8909440479bf926695e4b679d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34193 } } }
I20260812 07:07:50.910354 18462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:50.910354 18463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:50.911755 18486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:50.916437 18486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:50.923540 18489 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 07:07:50.926162 18459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:50.903063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:52999 (local address 127.0.0.1:34193)
0812 07:07:50.903235 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:50.903242 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:50.908638 (+ 5396us) server_negotiation.cc:407] Connection header received
0812 07:07:50.908842 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:50.908846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:50.908924 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:50.909035 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:50.910998 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:50.912288 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:50.914224 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:50.914950 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:50.915061 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:50.918496 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:50.918504 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:50.918510 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:50.918554 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:50.922366 (+ 3812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:50.923329 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:50.923338 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:50.923342 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:50.923504 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:50.924137 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:50.924142 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:50.924145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:50.924586 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:50.924856 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:50.925527 (+ 671us) server_negotiation.cc:299] Negotiation successful
0812 07:07:50.925703 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:07:50.926190 18486 catalog_manager.cc:1347] Generated new cluster ID: da0a61718d1f4791a3371363dc6b25ce
I20260812 07:07:50.926406 18486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:50.958573 18486 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:50.960332 18486 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:50.970917 18486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40aff7a8909440479bf926695e4b679d: Generated new TSK 0
I20260812 07:07:50.971802 18486 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:07:50.982470 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 18334
I20260812 07:07:50.983474 18447 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:51.074844 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 18334
W20260812 07:07:51.101027 18310 connection.cc:535] client connection to 127.0.0.1:34193 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34193): Connection reset by peer (error 104)
2026-08-12T07:07:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2186 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:07:51 dist-test-slave-lgr5 krb5kdc[18497](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:07:51 dist-test-slave-lgr5 krb5kdc[18497](info): set up 2 sockets
Aug 12 07:07:51 dist-test-slave-lgr5 krb5kdc[18497](info): commencing operation
krb5kdc: starting...
W20260812 07:07:52.332149 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:07:52 dist-test-slave-lgr5 krb5kdc[18497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:07:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:07:52.592794 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44477
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:52.913143 18518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:52.913426 18518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:52.913565 18518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:52.920643 18518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:52.920749 18518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:52.920832 18518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:52.920919 18518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:52.920979 18518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:52.930433 18518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44477
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:52.931658 18518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:52.932957 18518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:52.944053 18526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:52.944171 18534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:52.944141 18525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:52.945154 18518 server_base.cc:1034] running on GCE node
I20260812 07:07:52.945876 18518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:52.947109 18518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:52.948288 18518 hybrid_clock.cc:648] HybridClock initialized: now 1786518472948235 us; error 48 us; skew 500 ppm
I20260812 07:07:52.948614 18518 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:52 dist-test-slave-lgr5 krb5kdc[18497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518472, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:52.954229 18518 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:07:52.955519 18518 webserver.cc:460] Webserver started at http://127.0.0.1:42841/ using document root <none> and password file <none>
I20260812 07:07:52.956174 18518 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:52.956286 18518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:52.956533 18518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:52.959400 18518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "a8f3463b5b54437687dc50ebc500e17a"
format_stamp: "Formatted at 2026-08-12 07:07:52 on dist-test-slave-lgr5"
server_key: "f0163ee3fb41270503c03db457a7386e"
server_key_iv: "f7a435d9fa06417b3b8ea47c1d9994b8"
server_key_version: "encryptionkey@0"
I20260812 07:07:52.960127 18518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "a8f3463b5b54437687dc50ebc500e17a"
format_stamp: "Formatted at 2026-08-12 07:07:52 on dist-test-slave-lgr5"
server_key: "f0163ee3fb41270503c03db457a7386e"
server_key_iv: "f7a435d9fa06417b3b8ea47c1d9994b8"
server_key_version: "encryptionkey@0"
I20260812 07:07:52.964989 18518 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 07:07:52.968170 18549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:52.969373 18518 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:07:52.969523 18518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "a8f3463b5b54437687dc50ebc500e17a"
format_stamp: "Formatted at 2026-08-12 07:07:52 on dist-test-slave-lgr5"
server_key: "f0163ee3fb41270503c03db457a7386e"
server_key_iv: "f7a435d9fa06417b3b8ea47c1d9994b8"
server_key_version: "encryptionkey@0"
I20260812 07:07:52.969697 18518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:52.998205 18518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:53.002164 18518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:53.002415 18518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:53.019398 18518 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44477
I20260812 07:07:53.019447 18645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44477 every 8 connection(s)
I20260812 07:07:53.020726 18518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:53.024267 18646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:53.024363 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 18518
I20260812 07:07:53.024529 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:07:53.024945 12035 external_mini_cluster.cc:1428] Setting key da3c14c9d16b0d2f29ea179e7d8d1244
I20260812 07:07:53.038650 18646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a: Bootstrap starting.
I20260812 07:07:53.038841 18649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:53.026583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:48749 (local address 127.0.0.1:44477)
0812 07:07:53.027415 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:53.027429 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:53.027481 (+ 52us) server_negotiation.cc:407] Connection header received
0812 07:07:53.029222 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:53.029253 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:53.029737 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:53.030176 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:53.030659 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.032651 (+ 1992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.033938 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.034961 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.035179 (+ 218us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:53.036275 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:53.036286 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:53.036303 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:53.036947 (+ 644us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:53.037043 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:53.037072 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:53.037233 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:53.037487 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 07:07:53.037641 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":339,"threads_started":1}
I20260812 07:07:53.042088 18646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:53.042616 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:07:53.043171 18646 log.cc:824] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:53.045616 18646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a: No bootstrap required, opened a new log
I20260812 07:07:53.052923 18649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:53.046782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:47817 (local address 127.0.0.1:44477)
0812 07:07:53.046934 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:53.046939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:53.047256 (+ 317us) server_negotiation.cc:407] Connection header received
0812 07:07:53.047394 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:53.047400 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:53.047477 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:53.047580 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:53.048034 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.049162 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.050401 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:53.051082 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:53.051204 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:53.052191 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:53.052199 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:53.052204 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:53.052224 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:53.052274 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:53.052295 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:53.052408 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:53.052555 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 07:07:53.052604 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:07:53.054509 18646 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f3463b5b54437687dc50ebc500e17a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44477 } }
I20260812 07:07:53.054824 18646 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:53.054896 18646 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8f3463b5b54437687dc50ebc500e17a, State: Initialized, Role: FOLLOWER
I20260812 07:07:53.055361 18646 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f3463b5b54437687dc50ebc500e17a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44477 } }
I20260812 07:07:53.055568 18646 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:53.055665 18646 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:53.055781 18646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:53.057837 18646 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f3463b5b54437687dc50ebc500e17a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44477 } }
I20260812 07:07:53.058393 18646 leader_election.cc:302] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8f3463b5b54437687dc50ebc500e17a; no voters:
I20260812 07:07:53.058899 18646 leader_election.cc:288] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:53.059055 18658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:53.059661 18658 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a [term 1 LEADER]: Becoming Leader. State: Replica: a8f3463b5b54437687dc50ebc500e17a, State: Running, Role: LEADER
I20260812 07:07:53.060202 18658 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f3463b5b54437687dc50ebc500e17a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44477 } }
I20260812 07:07:53.060883 18646 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:53.062515 18661 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8f3463b5b54437687dc50ebc500e17a. Latest consensus state: current_term: 1 leader_uuid: "a8f3463b5b54437687dc50ebc500e17a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f3463b5b54437687dc50ebc500e17a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44477 } } }
I20260812 07:07:53.062804 18661 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:53.064352 18669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:53.064282 18659 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8f3463b5b54437687dc50ebc500e17a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f3463b5b54437687dc50ebc500e17a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44477 } } }
I20260812 07:07:53.064585 18659 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:53.068727 18669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:53.078047 18669 catalog_manager.cc:1347] Generated new cluster ID: 81d2fe8abf0544ebb15a73ca450b1a8a
I20260812 07:07:53.078140 18669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:53.103031 18669 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:53.104252 18669 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:53.111652 18669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8f3463b5b54437687dc50ebc500e17a: Generated new TSK 0
I20260812 07:07:53.112429 18669 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:07:53.125268 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 18518
W20260812 07:07:53.126242 18639 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.158:48749
W20260812 07:07:53.126950 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 18518: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:07:53.127082 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 18518
W20260812 07:07:53.153367 18495 connection.cc:535] client connection to 127.0.0.1:44477 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44477): Connection reset by peer (error 104)
2026-08-12T07:07:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2051 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:07:53 dist-test-slave-lgr5 krb5kdc[18696](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:07:53 dist-test-slave-lgr5 krb5kdc[18696](info): set up 2 sockets
Aug 12 07:07:53 dist-test-slave-lgr5 krb5kdc[18696](info): commencing operation
krb5kdc: starting...
W20260812 07:07:54.374722 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:07:54 dist-test-slave-lgr5 krb5kdc[18696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:07:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:07:54.636561 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39421
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:54.956324 18716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:54.956583 18716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:54.956688 18716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:54.963722 18716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:54.963821 18716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:54.963872 18716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:54.963932 18716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:54.963974 18716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:54.973546 18716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39421
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:54.974632 18716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:54.975860 18716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:54.986804 18722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:54.986886 18723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:54.986845 18726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:54.987651 18716 server_base.cc:1034] running on GCE node
I20260812 07:07:54.988366 18716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:54.989610 18716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:54.990796 18716 hybrid_clock.cc:648] HybridClock initialized: now 1786518474990755 us; error 30 us; skew 500 ppm
I20260812 07:07:54.991117 18716 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:54 dist-test-slave-lgr5 krb5kdc[18696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518474, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:54.994033 18716 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:07:54.995391 18716 webserver.cc:460] Webserver started at http://127.0.0.1:39773/ using document root <none> and password file <none>
I20260812 07:07:54.996011 18716 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:54.996145 18716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:54.996443 18716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:54.999275 18716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "e325dd226f784302b8a4013647bba040"
format_stamp: "Formatted at 2026-08-12 07:07:54 on dist-test-slave-lgr5"
server_key: "0b5a1bac1e5671bd4be65ac7b095efaa"
server_key_iv: "669a67abda047fd30bc6d397093304b6"
server_key_version: "encryptionkey@0"
I20260812 07:07:54.999974 18716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "e325dd226f784302b8a4013647bba040"
format_stamp: "Formatted at 2026-08-12 07:07:54 on dist-test-slave-lgr5"
server_key: "0b5a1bac1e5671bd4be65ac7b095efaa"
server_key_iv: "669a67abda047fd30bc6d397093304b6"
server_key_version: "encryptionkey@0"
I20260812 07:07:55.004784 18716 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:07:55.007990 18738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:55.009212 18716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 07:07:55.009353 18716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "e325dd226f784302b8a4013647bba040"
format_stamp: "Formatted at 2026-08-12 07:07:54 on dist-test-slave-lgr5"
server_key: "0b5a1bac1e5671bd4be65ac7b095efaa"
server_key_iv: "669a67abda047fd30bc6d397093304b6"
server_key_version: "encryptionkey@0"
I20260812 07:07:55.009521 18716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:55.014986 18716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:55.018715 18716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:55.018996 18716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:55.035668 18716 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39421
I20260812 07:07:55.035712 18840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39421 every 8 connection(s)
I20260812 07:07:55.036972 18716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:55.038380 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 18716
I20260812 07:07:55.038568 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:07:55.039099 12035 external_mini_cluster.cc:1428] Setting key 21703186347c5b9761cc70ed9abfc580
I20260812 07:07:55.041251 18841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:55.049348 18847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:55.040954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:58167 (local address 127.0.0.1:39421)
0812 07:07:55.043311 (+ 2357us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:55.043327 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:55.043659 (+ 332us) server_negotiation.cc:407] Connection header received
0812 07:07:55.045645 (+ 1986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:55.045673 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:55.046095 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:55.046535 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:55.047493 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:55.047502 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:55.047518 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:55.047535 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:55.047625 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:55.047653 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:55.047855 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:55.047991 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 07:07:55.048200 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2105,"thread_start_us":1865,"threads_started":1}
I20260812 07:07:55.052625 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:07:55.056552 18841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040: Bootstrap starting.
I20260812 07:07:55.059937 18847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:55.057711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:34743 (local address 127.0.0.1:39421)
0812 07:07:55.057886 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:55.057892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:55.058301 (+ 409us) server_negotiation.cc:407] Connection header received
0812 07:07:55.058410 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:55.058415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:55.058468 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:55.058585 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:55.059367 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:55.059372 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:55.059376 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:55.059392 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:55.059434 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:55.059454 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:55.059573 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:55.059618 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 07:07:55.059670 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:07:55.059940 18841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:55.061259 18841 log.cc:824] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:55.063751 18841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040: No bootstrap required, opened a new log
I20260812 07:07:55.071117 18841 raft_consensus.cc:348] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e325dd226f784302b8a4013647bba040" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39421 } }
I20260812 07:07:55.071440 18841 raft_consensus.cc:374] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:55.071516 18841 raft_consensus.cc:729] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e325dd226f784302b8a4013647bba040, State: Initialized, Role: FOLLOWER
I20260812 07:07:55.071985 18841 consensus_queue.cc:260] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e325dd226f784302b8a4013647bba040" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39421 } }
I20260812 07:07:55.072213 18841 raft_consensus.cc:388] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:55.072325 18841 raft_consensus.cc:482] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:55.072448 18841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:55.074471 18841 raft_consensus.cc:504] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e325dd226f784302b8a4013647bba040" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39421 } }
I20260812 07:07:55.074890 18841 leader_election.cc:302] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e325dd226f784302b8a4013647bba040; no voters:
I20260812 07:07:55.075393 18841 leader_election.cc:288] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:55.075558 18856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:55.076630 18856 raft_consensus.cc:686] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040 [term 1 LEADER]: Becoming Leader. State: Replica: e325dd226f784302b8a4013647bba040, State: Running, Role: LEADER
I20260812 07:07:55.077183 18856 consensus_queue.cc:237] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e325dd226f784302b8a4013647bba040" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39421 } }
I20260812 07:07:55.077471 18841 sys_catalog.cc:564] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:55.079669 18857 sys_catalog.cc:455] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e325dd226f784302b8a4013647bba040" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e325dd226f784302b8a4013647bba040" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39421 } } }
I20260812 07:07:55.079995 18857 sys_catalog.cc:458] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:55.080425 18858 sys_catalog.cc:455] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e325dd226f784302b8a4013647bba040. Latest consensus state: current_term: 1 leader_uuid: "e325dd226f784302b8a4013647bba040" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e325dd226f784302b8a4013647bba040" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39421 } } }
I20260812 07:07:55.080811 18874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:55.080816 18858 sys_catalog.cc:458] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:55.086226 18874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:55.094523 18874 catalog_manager.cc:1347] Generated new cluster ID: 614f1a48205741bdaa6ca2be3fdda314
I20260812 07:07:55.094619 18874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:55.117654 18874 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:55.118037 18874 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:55.125355 18874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e325dd226f784302b8a4013647bba040: Generated new TSK 0
I20260812 07:07:55.126163 18874 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:07:55.140370 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 18716
W20260812 07:07:55.141499 18835 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.158:58167
W20260812 07:07:55.142127 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 18716: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:07:55.142253 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 18716
2026-08-12T07:07:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2019 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:07:55 dist-test-slave-lgr5 krb5kdc[18891](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:07:55 dist-test-slave-lgr5 krb5kdc[18891](info): set up 2 sockets
Aug 12 07:07:55 dist-test-slave-lgr5 krb5kdc[18891](info): commencing operation
krb5kdc: starting...
W20260812 07:07:56.401147 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.007s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:07:56 dist-test-slave-lgr5 krb5kdc[18891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:07:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:07:56.660670 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:35973
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:34789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:35973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:56.979732 18922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:56.980014 18922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:56.980177 18922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:56.987079 18922 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:56.987195 18922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:56.987272 18922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:56.987358 18922 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:56.987421 18922 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:56.996862 18922 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:34789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:35973
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:35973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:56.998035 18922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:56.999254 18922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:57.010241 18932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:57.010192 18935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:57.010270 18933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:57.011353 18922 server_base.cc:1034] running on GCE node
I20260812 07:07:57.012108 18922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:57.013464 18922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:57.014662 18922 hybrid_clock.cc:648] HybridClock initialized: now 1786518477014616 us; error 42 us; skew 500 ppm
I20260812 07:07:57.014997 18922 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:57 dist-test-slave-lgr5 krb5kdc[18891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518477, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:57.017891 18922 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:07:57.019309 18922 webserver.cc:460] Webserver started at http://127.11.192.254:39753/ using document root <none> and password file <none>
I20260812 07:07:57.019935 18922 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:57.020059 18922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:57.020365 18922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:57.023187 18922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "7d5dc900c04348f7a828f6a32118d5bb"
format_stamp: "Formatted at 2026-08-12 07:07:57 on dist-test-slave-lgr5"
server_key: "bc0112f9a4f7f451e08aba8afe5a2026"
server_key_iv: "17b7c7f704efc6c57513f2d9eb12b5eb"
server_key_version: "encryptionkey@0"
I20260812 07:07:57.023896 18922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "7d5dc900c04348f7a828f6a32118d5bb"
format_stamp: "Formatted at 2026-08-12 07:07:57 on dist-test-slave-lgr5"
server_key: "bc0112f9a4f7f451e08aba8afe5a2026"
server_key_iv: "17b7c7f704efc6c57513f2d9eb12b5eb"
server_key_version: "encryptionkey@0"
I20260812 07:07:57.028724 18922 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:07:57.031949 18952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:57.033170 18922 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 07:07:57.033335 18922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "7d5dc900c04348f7a828f6a32118d5bb"
format_stamp: "Formatted at 2026-08-12 07:07:57 on dist-test-slave-lgr5"
server_key: "bc0112f9a4f7f451e08aba8afe5a2026"
server_key_iv: "17b7c7f704efc6c57513f2d9eb12b5eb"
server_key_version: "encryptionkey@0"
I20260812 07:07:57.033499 18922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:57.050608 18922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:57.054457 18922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:57.054703 18922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:57.070747 18922 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:35973
I20260812 07:07:57.070681 19058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:35973 every 8 connection(s)
I20260812 07:07:57.072001 18922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:57.075479 19060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:57.082002 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 18922
I20260812 07:07:57.082161 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:07:57.082558 12035 external_mini_cluster.cc:1428] Setting key 962b38d38eddde7bcaa090a0d4700a0c
I20260812 07:07:57.085983 19060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb: Bootstrap starting.
I20260812 07:07:57.089591 19060 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:57.090888 19060 log.cc:824] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:57.093492 19060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb: No bootstrap required, opened a new log
Aug 12 07:07:57 dist-test-slave-lgr5 krb5kdc[18891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:07:57.103188 19060 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d5dc900c04348f7a828f6a32118d5bb" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35973 } }
I20260812 07:07:57.103575 19060 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:57.103694 19060 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d5dc900c04348f7a828f6a32118d5bb, State: Initialized, Role: FOLLOWER
I20260812 07:07:57.104370 19060 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d5dc900c04348f7a828f6a32118d5bb" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35973 } }
I20260812 07:07:57.104674 19060 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:57.104818 19060 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:57.105018 19060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:57.106743 19064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:57.084211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38965 (local address 127.11.192.254:35973)
0812 07:07:57.084985 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:57.085000 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:57.085048 (+ 48us) server_negotiation.cc:407] Connection header received
0812 07:07:57.087319 (+ 2271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:57.087346 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:57.087869 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:57.088322 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:57.089632 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.091598 (+ 1966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.092902 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.093552 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.093699 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:57.098393 (+ 4694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:57.098402 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:57.098424 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:57.098473 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:57.102263 (+ 3790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:57.102986 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:57.102993 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:57.103001 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:57.103206 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:57.103638 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:57.103643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:57.103646 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:57.104336 (+ 690us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:57.104683 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:57.105109 (+ 426us) server_negotiation.cc:299] Negotiation successful
0812 07:07:57.105432 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":323,"threads_started":1}
I20260812 07:07:57.107811 19060 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d5dc900c04348f7a828f6a32118d5bb" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35973 } }
I20260812 07:07:57.108323 19060 leader_election.cc:302] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d5dc900c04348f7a828f6a32118d5bb; no voters:
I20260812 07:07:57.108901 19060 leader_election.cc:288] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:57.109000 19069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:57.110519 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:07:57.110457 19069 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb [term 1 LEADER]: Becoming Leader. State: Replica: 7d5dc900c04348f7a828f6a32118d5bb, State: Running, Role: LEADER
I20260812 07:07:57.110931 19069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d5dc900c04348f7a828f6a32118d5bb" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35973 } }
I20260812 07:07:57.111114 19060 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:57.113353 19074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d5dc900c04348f7a828f6a32118d5bb. Latest consensus state: current_term: 1 leader_uuid: "7d5dc900c04348f7a828f6a32118d5bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d5dc900c04348f7a828f6a32118d5bb" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35973 } } }
I20260812 07:07:57.113375 19072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d5dc900c04348f7a828f6a32118d5bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d5dc900c04348f7a828f6a32118d5bb" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 35973 } } }
I20260812 07:07:57.113811 19072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:57.113809 19074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:57.128925 19081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:57.133628 19081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:57.152773 19081 catalog_manager.cc:1347] Generated new cluster ID: 4e43241783bd4e9694d8280f30a5d380
I20260812 07:07:57.152884 19081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:57.159196 19064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:57.137516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38357 (local address 127.11.192.254:35973)
0812 07:07:57.137778 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:57.137785 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:57.140669 (+ 2884us) server_negotiation.cc:407] Connection header received
0812 07:07:57.140854 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:57.140860 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:57.140951 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:57.141071 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:57.143402 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.144703 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.146914 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:57.147703 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:57.147822 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:57.151439 (+ 3617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:57.151446 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:57.151452 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:07:57.151494 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:57.155422 (+ 3928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:57.156461 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:57.156468 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:57.156472 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:57.156627 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:07:57.157191 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:07:57.157197 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:07:57.157201 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:07:57.157628 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:57.157901 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:57.158593 (+ 692us) server_negotiation.cc:299] Negotiation successful
0812 07:07:57.158789 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260812 07:07:57.169062 19081 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:57.170775 19081 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:57.191385 19081 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d5dc900c04348f7a828f6a32118d5bb: Generated new TSK 0
I20260812 07:07:57.192205 19081 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:07:57.199244 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 18922
I20260812 07:07:57.200431 19052 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:07:57.289945 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 18922
W20260812 07:07:57.314239 18889 connection.cc:535] client connection to 127.11.192.254:35973 recv error: Network error: failed to read from TLS socket (remote: 127.11.192.254:35973): Connection reset by peer (error 104)
2026-08-12T07:07:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2141 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:07:57 dist-test-slave-lgr5 krb5kdc[19121](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:07:57 dist-test-slave-lgr5 krb5kdc[19121](info): set up 2 sockets
Aug 12 07:07:57 dist-test-slave-lgr5 krb5kdc[19121](info): commencing operation
krb5kdc: starting...
W20260812 07:07:58.553587 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:07:58 dist-test-slave-lgr5 krb5kdc[19121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:07:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:07:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:07:58.819422 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:32861
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:44973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:32861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:07:59.144896 19141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:07:59.145157 19141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:07:59.145282 19141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:07:59.152143 19141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:07:59.152242 19141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:07:59.152315 19141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:07:59.152400 19141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:07:59.152457 19141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:07:59.161717 19141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:44973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:32861
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:32861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:07:59.162842 19141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:07:59.164098 19141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:07:59.175515 19157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:59.175621 19158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:07:59.175647 19162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:07:59.176538 19141 server_base.cc:1034] running on GCE node
I20260812 07:07:59.177233 19141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:07:59.178470 19141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:07:59.179678 19141 hybrid_clock.cc:648] HybridClock initialized: now 1786518479179636 us; error 28 us; skew 500 ppm
I20260812 07:07:59.179987 19141 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:07:59 dist-test-slave-lgr5 krb5kdc[19121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518479, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:07:59.185649 19141 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:07:59.186932 19141 webserver.cc:460] Webserver started at http://127.11.192.254:34187/ using document root <none> and password file <none>
I20260812 07:07:59.187528 19141 fs_manager.cc:359] Metadata directory not provided
I20260812 07:07:59.187620 19141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:07:59.187855 19141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:07:59.190625 19141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "8851cb30343c42148f2da0d40b29e830"
format_stamp: "Formatted at 2026-08-12 07:07:59 on dist-test-slave-lgr5"
server_key: "65b375d7a658f38b6a39e9ae831732ec"
server_key_iv: "cb22f403f5b25fd9068013d02c902853"
server_key_version: "encryptionkey@0"
I20260812 07:07:59.191301 19141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "8851cb30343c42148f2da0d40b29e830"
format_stamp: "Formatted at 2026-08-12 07:07:59 on dist-test-slave-lgr5"
server_key: "65b375d7a658f38b6a39e9ae831732ec"
server_key_iv: "cb22f403f5b25fd9068013d02c902853"
server_key_version: "encryptionkey@0"
I20260812 07:07:59.196008 19141 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 07:07:59.199154 19172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:07:59.200368 19141 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260812 07:07:59.200516 19141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "8851cb30343c42148f2da0d40b29e830"
format_stamp: "Formatted at 2026-08-12 07:07:59 on dist-test-slave-lgr5"
server_key: "65b375d7a658f38b6a39e9ae831732ec"
server_key_iv: "cb22f403f5b25fd9068013d02c902853"
server_key_version: "encryptionkey@0"
I20260812 07:07:59.200673 19141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:07:59.217645 19141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:07:59.221012 19141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:07:59.221237 19141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:07:59.237906 19141 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:32861
I20260812 07:07:59.237939 19256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:32861 every 8 connection(s)
I20260812 07:07:59.239149 19141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:07:59.242614 19261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:07:59.242688 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 19141
I20260812 07:07:59.242874 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:07:59.243278 12035 external_mini_cluster.cc:1428] Setting key 4f995ffd8c72d9a14013c384a93d18c6
I20260812 07:07:59.257390 19264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:59.244930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51513 (local address 127.11.192.254:32861)
0812 07:07:59.245777 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:59.245795 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:59.245842 (+ 47us) server_negotiation.cc:407] Connection header received
0812 07:07:59.247538 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:59.247565 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:59.248108 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:59.248592 (+ 484us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:59.249100 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.251103 (+ 2003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.252441 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.253463 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.253663 (+ 200us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:59.254781 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:59.254791 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:59.254808 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:59.255530 (+ 722us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:59.255627 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:59.255657 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:59.255811 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:59.256020 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 07:07:59.256203 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":341,"threads_started":1}
I20260812 07:07:59.257490 19261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830: Bootstrap starting.
I20260812 07:07:59.260505 19261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830: Neither blocks nor log segments found. Creating new log.
I20260812 07:07:59.260903 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:07:59.261603 19261 log.cc:824] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830: Log is configured to *not* fsync() on all Append() calls
I20260812 07:07:59.264007 19261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830: No bootstrap required, opened a new log
I20260812 07:07:59.271714 19264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:07:59.265184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33497 (local address 127.11.192.254:32861)
0812 07:07:59.265402 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 07:07:59.265408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:07:59.265782 (+ 374us) server_negotiation.cc:407] Connection header received
0812 07:07:59.265921 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:07:59.265924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:07:59.265979 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:07:59.266060 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:07:59.266638 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.267400 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.269244 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:07:59.269769 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:07:59.269869 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:07:59.270979 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:07:59.270983 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:07:59.270990 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:07:59.271006 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:07:59.271060 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:07:59.271074 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:07:59.271173 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 07:07:59.271350 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 07:07:59.271388 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:07:59.273113 19261 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8851cb30343c42148f2da0d40b29e830" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 32861 } }
I20260812 07:07:59.273458 19261 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:07:59.273541 19261 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8851cb30343c42148f2da0d40b29e830, State: Initialized, Role: FOLLOWER
I20260812 07:07:59.274039 19261 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8851cb30343c42148f2da0d40b29e830" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 32861 } }
I20260812 07:07:59.274258 19261 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:07:59.274379 19261 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:07:59.274529 19261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:07:59.276638 19261 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8851cb30343c42148f2da0d40b29e830" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 32861 } }
I20260812 07:07:59.277108 19261 leader_election.cc:302] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8851cb30343c42148f2da0d40b29e830; no voters:
I20260812 07:07:59.277613 19261 leader_election.cc:288] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:07:59.277719 19283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:07:59.278723 19283 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830 [term 1 LEADER]: Becoming Leader. State: Replica: 8851cb30343c42148f2da0d40b29e830, State: Running, Role: LEADER
I20260812 07:07:59.279603 19283 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8851cb30343c42148f2da0d40b29e830" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 32861 } }
I20260812 07:07:59.279760 19261 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:07:59.282661 19284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8851cb30343c42148f2da0d40b29e830" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8851cb30343c42148f2da0d40b29e830" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 32861 } } }
I20260812 07:07:59.282958 19284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:59.283811 19297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:07:59.283628 19285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8851cb30343c42148f2da0d40b29e830. Latest consensus state: current_term: 1 leader_uuid: "8851cb30343c42148f2da0d40b29e830" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8851cb30343c42148f2da0d40b29e830" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 32861 } } }
I20260812 07:07:59.283984 19285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830 [sys.catalog]: This master's current role is: LEADER
I20260812 07:07:59.289147 19297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:07:59.297281 19297 catalog_manager.cc:1347] Generated new cluster ID: 81f31e4b7c074c15b151088fdb18cb20
I20260812 07:07:59.297376 19297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:07:59.306821 19297 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:07:59.308461 19297 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:07:59.326977 19297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8851cb30343c42148f2da0d40b29e830: Generated new TSK 0
I20260812 07:07:59.327893 19297 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:07:59.338317 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 19141
W20260812 07:07:59.339308 19249 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51513
W20260812 07:07:59.339977 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 19141: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:07:59.340157 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 19141
W20260812 07:07:59.365572 19119 connection.cc:535] client connection to 127.11.192.254:32861 recv error: Network error: failed to read from TLS socket (remote: 127.11.192.254:32861): Connection reset by peer (error 104)
2026-08-12T07:07:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2051 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:07:59 dist-test-slave-lgr5 krb5kdc[19327](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:07:59 dist-test-slave-lgr5 krb5kdc[19327](info): set up 2 sockets
Aug 12 07:07:59 dist-test-slave-lgr5 krb5kdc[19327](info): commencing operation
krb5kdc: starting...
W20260812 07:08:00.587348 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:08:00 dist-test-slave-lgr5 krb5kdc[19327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:08:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:08:00.846730 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:33839
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:44613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:33839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:01.165798 19346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:01.166065 19346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:01.166201 19346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:01.173553 19346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:01.173655 19346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:01.173730 19346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:01.173812 19346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:01.173875 19346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:01.183372 19346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:44613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:33839
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:33839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:08:01.184584 19346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:01.185822 19346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:01.196727 19352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:01.196787 19355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:01.196947 19357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:01.197832 19346 server_base.cc:1034] running on GCE node
I20260812 07:08:01.198504 19346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:01.199730 19346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:01.200939 19346 hybrid_clock.cc:648] HybridClock initialized: now 1786518481200894 us; error 29 us; skew 500 ppm
I20260812 07:08:01.201252 19346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:08:01 dist-test-slave-lgr5 krb5kdc[19327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518481, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:01.204178 19346 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:08:01.205531 19346 webserver.cc:460] Webserver started at http://127.11.192.254:44259/ using document root <none> and password file <none>
I20260812 07:08:01.206159 19346 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:01.206278 19346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:01.206568 19346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:01.209461 19346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "c26de5fb3d9f4b52a4000242476772db"
format_stamp: "Formatted at 2026-08-12 07:08:01 on dist-test-slave-lgr5"
server_key: "6ce00d651f4c76ea9add13b148eaffd8"
server_key_iv: "4a5d36c8d92124f4a537e367151f6639"
server_key_version: "encryptionkey@0"
I20260812 07:08:01.210191 19346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "c26de5fb3d9f4b52a4000242476772db"
format_stamp: "Formatted at 2026-08-12 07:08:01 on dist-test-slave-lgr5"
server_key: "6ce00d651f4c76ea9add13b148eaffd8"
server_key_iv: "4a5d36c8d92124f4a537e367151f6639"
server_key_version: "encryptionkey@0"
I20260812 07:08:01.215277 19346 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:08:01.218492 19372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:01.219708 19346 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:08:01.219866 19346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "c26de5fb3d9f4b52a4000242476772db"
format_stamp: "Formatted at 2026-08-12 07:08:01 on dist-test-slave-lgr5"
server_key: "6ce00d651f4c76ea9add13b148eaffd8"
server_key_iv: "4a5d36c8d92124f4a537e367151f6639"
server_key_version: "encryptionkey@0"
I20260812 07:08:01.220038 19346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:01.225389 19346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:01.229003 19346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:01.229261 19346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:01.245908 19346 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:33839
I20260812 07:08:01.245941 19466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:33839 every 8 connection(s)
I20260812 07:08:01.247140 19346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:01.247386 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 19346
I20260812 07:08:01.247548 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:08:01.248010 12035 external_mini_cluster.cc:1428] Setting key 46ca274f35665cc0b0f7399b62c0d5f2
I20260812 07:08:01.251200 19467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:01.255192 19473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:01.249681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59203 (local address 127.11.192.254:33839)
0812 07:08:01.250429 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:01.250441 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:01.250480 (+ 39us) server_negotiation.cc:407] Connection header received
0812 07:08:01.252113 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:01.252143 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:01.252452 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:01.252749 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:01.253756 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:01.253762 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:01.253774 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:01.253791 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:01.253869 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:01.253888 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:01.254067 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:01.254162 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 07:08:01.254328 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":346,"threads_started":1}
I20260812 07:08:01.259044 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:08:01.264015 19467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db: Bootstrap starting.
I20260812 07:08:01.266767 19473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:01.264377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45675 (local address 127.11.192.254:33839)
0812 07:08:01.264570 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:01.264576 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:01.264908 (+ 332us) server_negotiation.cc:407] Connection header received
0812 07:08:01.265053 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:01.265056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:01.265103 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:01.265188 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:01.266140 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:01.266145 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:01.266148 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:01.266164 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:01.266203 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:01.266220 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:01.266318 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:01.266431 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 07:08:01.266475 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 07:08:01.267670 19467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:01.268756 19467 log.cc:824] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:01.271318 19467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db: No bootstrap required, opened a new log
I20260812 07:08:01.278553 19467 raft_consensus.cc:348] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c26de5fb3d9f4b52a4000242476772db" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33839 } }
I20260812 07:08:01.278865 19467 raft_consensus.cc:374] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:01.278936 19467 raft_consensus.cc:729] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c26de5fb3d9f4b52a4000242476772db, State: Initialized, Role: FOLLOWER
I20260812 07:08:01.279410 19467 consensus_queue.cc:260] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c26de5fb3d9f4b52a4000242476772db" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33839 } }
I20260812 07:08:01.279601 19467 raft_consensus.cc:388] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:08:01.279701 19467 raft_consensus.cc:482] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:08:01.279822 19467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:01.281824 19467 raft_consensus.cc:504] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c26de5fb3d9f4b52a4000242476772db" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33839 } }
I20260812 07:08:01.282241 19467 leader_election.cc:302] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c26de5fb3d9f4b52a4000242476772db; no voters:
I20260812 07:08:01.282716 19467 leader_election.cc:288] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:08:01.282840 19489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:01.283437 19489 raft_consensus.cc:686] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db [term 1 LEADER]: Becoming Leader. State: Replica: c26de5fb3d9f4b52a4000242476772db, State: Running, Role: LEADER
I20260812 07:08:01.283957 19489 consensus_queue.cc:237] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c26de5fb3d9f4b52a4000242476772db" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33839 } }
I20260812 07:08:01.284590 19467 sys_catalog.cc:564] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:01.286718 19491 sys_catalog.cc:455] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c26de5fb3d9f4b52a4000242476772db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c26de5fb3d9f4b52a4000242476772db" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33839 } } }
I20260812 07:08:01.287138 19491 sys_catalog.cc:458] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:01.287407 19494 sys_catalog.cc:455] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db [sys.catalog]: SysCatalogTable state changed. Reason: New leader c26de5fb3d9f4b52a4000242476772db. Latest consensus state: current_term: 1 leader_uuid: "c26de5fb3d9f4b52a4000242476772db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c26de5fb3d9f4b52a4000242476772db" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 33839 } } }
I20260812 07:08:01.287751 19494 sys_catalog.cc:458] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:01.289544 19503 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:01.294303 19503 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:01.302522 19503 catalog_manager.cc:1347] Generated new cluster ID: 38011d07f2984ca29f12effd62cfe78d
I20260812 07:08:01.302654 19503 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:01.321462 19503 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:01.321815 19503 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:01.334923 19503 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c26de5fb3d9f4b52a4000242476772db: Generated new TSK 0
I20260812 07:08:01.335851 19503 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:08:01.349260 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 19346
W20260812 07:08:01.350277 19459 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59203
W20260812 07:08:01.350914 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 19346: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:08:01.351037 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 19346
2026-08-12T07:08:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2014 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:08:01 dist-test-slave-lgr5 krb5kdc[19523](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:08:01 dist-test-slave-lgr5 krb5kdc[19523](info): set up 2 sockets
Aug 12 07:08:01 dist-test-slave-lgr5 krb5kdc[19523](info): commencing operation
krb5kdc: starting...
W20260812 07:08:02.647766 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.007s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:08:02 dist-test-slave-lgr5 krb5kdc[19523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:08:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:08:02.917304 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:43115
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:44465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:43115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:03.243809 19541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:03.244123 19541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:03.244282 19541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:03.251339 19541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:03.251459 19541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:03.251549 19541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:03.251646 19541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:03.251717 19541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:03.261207 19541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:44465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:43115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:43115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:08:03.262360 19541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:03.263639 19541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:03.274863 19549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:03.274976 19550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:03.274910 19555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:03.275857 19541 server_base.cc:1034] running on GCE node
I20260812 07:08:03.276615 19541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:03.277977 19541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:03.279177 19541 hybrid_clock.cc:648] HybridClock initialized: now 1786518483279141 us; error 36 us; skew 500 ppm
I20260812 07:08:03.279496 19541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:08:03 dist-test-slave-lgr5 krb5kdc[19523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518483, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:03.282513 19541 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:08:03.283897 19541 webserver.cc:460] Webserver started at http://127.11.192.254:33025/ using document root <none> and password file <none>
I20260812 07:08:03.284564 19541 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:03.284679 19541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:03.284956 19541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:03.287758 19541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "901038be6dd9482aa401eab149dded92"
format_stamp: "Formatted at 2026-08-12 07:08:03 on dist-test-slave-lgr5"
server_key: "5f1df3fb9ccd3d233d39176d8bb92a5c"
server_key_iv: "e3ff0959c11e67aa34bb1b9d83c53011"
server_key_version: "encryptionkey@0"
I20260812 07:08:03.288496 19541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "901038be6dd9482aa401eab149dded92"
format_stamp: "Formatted at 2026-08-12 07:08:03 on dist-test-slave-lgr5"
server_key: "5f1df3fb9ccd3d233d39176d8bb92a5c"
server_key_iv: "e3ff0959c11e67aa34bb1b9d83c53011"
server_key_version: "encryptionkey@0"
I20260812 07:08:03.293219 19541 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:08:03.296377 19564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:03.297605 19541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:03.297755 19541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "901038be6dd9482aa401eab149dded92"
format_stamp: "Formatted at 2026-08-12 07:08:03 on dist-test-slave-lgr5"
server_key: "5f1df3fb9ccd3d233d39176d8bb92a5c"
server_key_iv: "e3ff0959c11e67aa34bb1b9d83c53011"
server_key_version: "encryptionkey@0"
I20260812 07:08:03.297919 19541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:03.309113 19541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:03.312554 19541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:03.312788 19541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:03.329455 19541 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:43115
I20260812 07:08:03.329463 19658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:43115 every 8 connection(s)
I20260812 07:08:03.330683 19541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:03.334156 19660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:03.338450 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 19541
I20260812 07:08:03.338634 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:08:03.339180 12035 external_mini_cluster.cc:1428] Setting key 7537d9d1b6e7170917133d47a1930076
I20260812 07:08:03.345001 19660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92: Bootstrap starting.
I20260812 07:08:03.348285 19660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:03.349422 19660 log.cc:824] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:03.352519 19660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92: No bootstrap required, opened a new log
Aug 12 07:08:03 dist-test-slave-lgr5 krb5kdc[19523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:08:03.360008 19660 raft_consensus.cc:348] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901038be6dd9482aa401eab149dded92" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43115 } }
I20260812 07:08:03.360410 19660 raft_consensus.cc:374] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:03.360510 19660 raft_consensus.cc:729] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 901038be6dd9482aa401eab149dded92, State: Initialized, Role: FOLLOWER
I20260812 07:08:03.361089 19660 consensus_queue.cc:260] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901038be6dd9482aa401eab149dded92" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43115 } }
I20260812 07:08:03.361313 19660 raft_consensus.cc:388] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:08:03.361377 19665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:03.340872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57285 (local address 127.11.192.254:43115)
0812 07:08:03.341591 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:03.341602 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:03.341644 (+ 42us) server_negotiation.cc:407] Connection header received
0812 07:08:03.343595 (+ 1951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:03.343618 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:03.343958 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:03.344286 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:03.346051 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:03.347355 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:03.349104 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:03.349820 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:03.350012 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:03.354425 (+ 4413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:03.354435 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:03.354448 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:03.354481 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:03.357516 (+ 3035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:03.358166 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:03.358174 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:03.358183 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:03.358368 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:03.358755 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:03.358759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:03.358761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:03.359250 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:03.359557 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:03.359972 (+ 415us) server_negotiation.cc:299] Negotiation successful
0812 07:08:03.360420 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":325,"threads_started":1}
I20260812 07:08:03.361456 19660 raft_consensus.cc:482] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:08:03.361615 19660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:03.364338 19660 raft_consensus.cc:504] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901038be6dd9482aa401eab149dded92" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43115 } }
I20260812 07:08:03.364867 19660 leader_election.cc:302] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 901038be6dd9482aa401eab149dded92; no voters:
I20260812 07:08:03.365006 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:08:03.365454 19660 leader_election.cc:288] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:08:03.365585 19667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:03.367156 19660 sys_catalog.cc:564] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:03.371299 19667 raft_consensus.cc:686] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92 [term 1 LEADER]: Becoming Leader. State: Replica: 901038be6dd9482aa401eab149dded92, State: Running, Role: LEADER
I20260812 07:08:03.371975 19667 consensus_queue.cc:237] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901038be6dd9482aa401eab149dded92" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43115 } }
I20260812 07:08:03.374918 19670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 901038be6dd9482aa401eab149dded92. Latest consensus state: current_term: 1 leader_uuid: "901038be6dd9482aa401eab149dded92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901038be6dd9482aa401eab149dded92" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43115 } } }
I20260812 07:08:03.375286 19670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:03.375705 19668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "901038be6dd9482aa401eab149dded92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901038be6dd9482aa401eab149dded92" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 43115 } } }
I20260812 07:08:03.376003 19668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:03.379207 19696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:03.384337 19696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:03.392231 19665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:03.374447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34869 (local address 127.11.192.254:43115)
0812 07:08:03.375078 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:03.375085 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:03.375621 (+ 536us) server_negotiation.cc:407] Connection header received
0812 07:08:03.375878 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:03.375883 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:03.375958 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:03.376030 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:03.378240 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:03.379316 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:03.381052 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:03.381728 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:03.381855 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:03.384765 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:03.384772 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:03.384777 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:03.384813 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:03.388448 (+ 3635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:03.389245 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:03.389250 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:03.389252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:03.389424 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:03.389910 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:03.389916 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:03.389920 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:03.390323 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:03.390544 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:03.391585 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0812 07:08:03.391812 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496}
I20260812 07:08:03.398276 19696 catalog_manager.cc:1347] Generated new cluster ID: 58aa07f3c7854d979e5fe82bb8e1678e
I20260812 07:08:03.398388 19696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:03.423664 19696 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:03.425491 19696 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:03.434720 19696 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 901038be6dd9482aa401eab149dded92: Generated new TSK 0
I20260812 07:08:03.435678 19696 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:08:03.446281 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 19541
I20260812 07:08:03.447414 19651 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:08:03.535866 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 19541
2026-08-12T07:08:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2177 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:08:03 dist-test-slave-lgr5 krb5kdc[19710](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:08:03 dist-test-slave-lgr5 krb5kdc[19710](info): set up 2 sockets
Aug 12 07:08:03 dist-test-slave-lgr5 krb5kdc[19710](info): commencing operation
krb5kdc: starting...
W20260812 07:08:04.780625 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.010s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:08:04 dist-test-slave-lgr5 krb5kdc[19710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:08:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:08:05.046648 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:39129
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:36259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:39129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:05.367238 19732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:05.367501 19732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:05.367605 19732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:05.374504 19732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:05.374608 19732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:05.374660 19732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:05.374720 19732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:05.374782 19732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:05.384167 19732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:36259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:39129
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:39129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:08:05.385299 19732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:05.386503 19732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:05.397306 19740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:05.397408 19745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:05.397378 19741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:05.398240 19732 server_base.cc:1034] running on GCE node
I20260812 07:08:05.398945 19732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:05.400250 19732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:05.401458 19732 hybrid_clock.cc:648] HybridClock initialized: now 1786518485401411 us; error 38 us; skew 500 ppm
I20260812 07:08:05.401786 19732 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:08:05 dist-test-slave-lgr5 krb5kdc[19710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518485, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:05.412721 19732 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:08:05.414096 19732 webserver.cc:460] Webserver started at http://127.11.192.254:35525/ using document root <none> and password file <none>
I20260812 07:08:05.414710 19732 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:05.414808 19732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:05.415057 19732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:05.417935 19732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "d915fd2de21a4af4aa83f0181cab86d1"
format_stamp: "Formatted at 2026-08-12 07:08:05 on dist-test-slave-lgr5"
server_key: "2e943989e75eebfc4f0a410a64ad5fad"
server_key_iv: "018234e19f6384fcbb7de44b990399f3"
server_key_version: "encryptionkey@0"
I20260812 07:08:05.418637 19732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "d915fd2de21a4af4aa83f0181cab86d1"
format_stamp: "Formatted at 2026-08-12 07:08:05 on dist-test-slave-lgr5"
server_key: "2e943989e75eebfc4f0a410a64ad5fad"
server_key_iv: "018234e19f6384fcbb7de44b990399f3"
server_key_version: "encryptionkey@0"
I20260812 07:08:05.423496 19732 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:08:05.426770 19761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:05.427965 19732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:08:05.428133 19732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "d915fd2de21a4af4aa83f0181cab86d1"
format_stamp: "Formatted at 2026-08-12 07:08:05 on dist-test-slave-lgr5"
server_key: "2e943989e75eebfc4f0a410a64ad5fad"
server_key_iv: "018234e19f6384fcbb7de44b990399f3"
server_key_version: "encryptionkey@0"
I20260812 07:08:05.428297 19732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:05.448009 19732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:05.451594 19732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:05.451817 19732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:05.468501 19732 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:39129
I20260812 07:08:05.468521 19852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:39129 every 8 connection(s)
I20260812 07:08:05.469705 19732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:05.473124 19853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:05.478080 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 19732
I20260812 07:08:05.478237 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:08:05.478627 12035 external_mini_cluster.cc:1428] Setting key 04be13a3cd74c1d665206b204e877587
I20260812 07:08:05.483181 19853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1: Bootstrap starting.
I20260812 07:08:05.486449 19853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:05.487524 19853 log.cc:824] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:05.490101 19853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1: No bootstrap required, opened a new log
I20260812 07:08:05.490906 19856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:05.480220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40137 (local address 127.11.192.254:39129)
0812 07:08:05.480975 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:05.480986 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:05.481046 (+ 60us) server_negotiation.cc:407] Connection header received
0812 07:08:05.482668 (+ 1622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:05.482690 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:05.483032 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:05.483346 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:05.483888 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:05.485203 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:05.486963 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:05.487605 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:05.487773 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:05.488889 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:05.488897 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:05.488909 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:05.489376 (+ 467us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:05.489444 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:05.489469 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:05.489575 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:05.489789 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 07:08:05.490020 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":339,"threads_started":1}
I20260812 07:08:05.494184 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:08:05.497466 19853 raft_consensus.cc:348] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d915fd2de21a4af4aa83f0181cab86d1" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39129 } }
I20260812 07:08:05.497773 19853 raft_consensus.cc:374] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:05.497893 19853 raft_consensus.cc:729] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d915fd2de21a4af4aa83f0181cab86d1, State: Initialized, Role: FOLLOWER
I20260812 07:08:05.498555 19853 consensus_queue.cc:260] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d915fd2de21a4af4aa83f0181cab86d1" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39129 } }
I20260812 07:08:05.498821 19853 raft_consensus.cc:388] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:08:05.498962 19853 raft_consensus.cc:482] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:08:05.499125 19853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:05.501248 19853 raft_consensus.cc:504] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d915fd2de21a4af4aa83f0181cab86d1" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39129 } }
I20260812 07:08:05.501684 19853 leader_election.cc:302] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d915fd2de21a4af4aa83f0181cab86d1; no voters:
I20260812 07:08:05.502272 19853 leader_election.cc:288] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:08:05.502368 19866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:05.503046 19866 raft_consensus.cc:686] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1 [term 1 LEADER]: Becoming Leader. State: Replica: d915fd2de21a4af4aa83f0181cab86d1, State: Running, Role: LEADER
I20260812 07:08:05.503569 19866 consensus_queue.cc:237] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d915fd2de21a4af4aa83f0181cab86d1" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39129 } }
I20260812 07:08:05.504101 19856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:05.498283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47565 (local address 127.11.192.254:39129)
0812 07:08:05.498416 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:05.498422 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:05.498797 (+ 375us) server_negotiation.cc:407] Connection header received
0812 07:08:05.498962 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:05.498966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:05.499025 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:05.499110 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:05.499682 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:05.500439 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:05.501710 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:05.502204 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:05.502331 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:05.503344 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:05.503351 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:05.503355 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:05.503379 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:05.503424 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:05.503443 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:05.503535 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:05.503708 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 07:08:05.503793 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 07:08:05.504635 19853 sys_catalog.cc:564] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:05.506520 19867 sys_catalog.cc:455] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d915fd2de21a4af4aa83f0181cab86d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d915fd2de21a4af4aa83f0181cab86d1" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39129 } } }
I20260812 07:08:05.506906 19867 sys_catalog.cc:458] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:05.508285 19868 sys_catalog.cc:455] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d915fd2de21a4af4aa83f0181cab86d1. Latest consensus state: current_term: 1 leader_uuid: "d915fd2de21a4af4aa83f0181cab86d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d915fd2de21a4af4aa83f0181cab86d1" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39129 } } }
I20260812 07:08:05.508510 19868 sys_catalog.cc:458] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1 [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:05.509107 19877 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:05.513023 19877 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:05.520798 19877 catalog_manager.cc:1347] Generated new cluster ID: 329f29a0ca2e4793b92be4b3fe270188
I20260812 07:08:05.520896 19877 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:05.533361 19877 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:05.534469 19877 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:05.543179 19877 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d915fd2de21a4af4aa83f0181cab86d1: Generated new TSK 0
I20260812 07:08:05.544044 19877 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:08:05.555418 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 19732
W20260812 07:08:05.556362 19847 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40137
W20260812 07:08:05.556958 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 19732: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:08:05.557081 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 19732
2026-08-12T07:08:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2023 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:08:05 dist-test-slave-lgr5 krb5kdc[19902](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:08:05 dist-test-slave-lgr5 krb5kdc[19902](info): set up 2 sockets
Aug 12 07:08:05 dist-test-slave-lgr5 krb5kdc[19902](info): commencing operation
krb5kdc: starting...
W20260812 07:08:06.809574 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.004s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:08:06 dist-test-slave-lgr5 krb5kdc[19902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:08:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:08:07.075897 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:40519
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:35821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:40519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:07.401875 19929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:07.402158 19929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:07.402307 19929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:07.409387 19929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:07.409498 19929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:07.409581 19929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:07.409674 19929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:07.409739 19929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:07.419788 19929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:35821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:40519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:40519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:08:07.420989 19929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:07.422271 19929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:07.433604 19941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:07.433676 19939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:07.433710 19938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:07.434641 19929 server_base.cc:1034] running on GCE node
I20260812 07:08:07.435348 19929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:07.436666 19929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:07.437904 19929 hybrid_clock.cc:648] HybridClock initialized: now 1786518487437863 us; error 31 us; skew 500 ppm
I20260812 07:08:07.438227 19929 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:08:07 dist-test-slave-lgr5 krb5kdc[19902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518487, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:07.441098 19929 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:08:07.442405 19929 webserver.cc:460] Webserver started at http://127.11.192.254:46347/ using document root <none> and password file <none>
I20260812 07:08:07.443014 19929 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:07.443122 19929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:07.443398 19929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:07.446193 19929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "918ad01804b6408db91adc8af89a7ede"
format_stamp: "Formatted at 2026-08-12 07:08:07 on dist-test-slave-lgr5"
server_key: "52acddeff7456d402f5bf3ee356d24ce"
server_key_iv: "41ff66ccd9abeb0ce466ee4ce535ae12"
server_key_version: "encryptionkey@0"
I20260812 07:08:07.446928 19929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "918ad01804b6408db91adc8af89a7ede"
format_stamp: "Formatted at 2026-08-12 07:08:07 on dist-test-slave-lgr5"
server_key: "52acddeff7456d402f5bf3ee356d24ce"
server_key_iv: "41ff66ccd9abeb0ce466ee4ce535ae12"
server_key_version: "encryptionkey@0"
I20260812 07:08:07.451673 19929 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 07:08:07.454936 19958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:07.456175 19929 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:08:07.456329 19929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "918ad01804b6408db91adc8af89a7ede"
format_stamp: "Formatted at 2026-08-12 07:08:07 on dist-test-slave-lgr5"
server_key: "52acddeff7456d402f5bf3ee356d24ce"
server_key_iv: "41ff66ccd9abeb0ce466ee4ce535ae12"
server_key_version: "encryptionkey@0"
I20260812 07:08:07.456504 19929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:07.461807 19929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:07.465276 19929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:07.465534 19929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:07.482203 19929 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:40519
I20260812 07:08:07.482240 20051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:40519 every 8 connection(s)
I20260812 07:08:07.483407 19929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:07.486991 20052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:07.490332 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 19929
I20260812 07:08:07.490482 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:08:07.490895 12035 external_mini_cluster.cc:1428] Setting key 7886f7c5dd6f476a0571d9c41f470ee4
I20260812 07:08:07.498807 20059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:07.492478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38291 (local address 127.11.192.254:40519)
0812 07:08:07.493306 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:07.493332 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:07.493381 (+ 49us) server_negotiation.cc:407] Connection header received
0812 07:08:07.495195 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:07.495223 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:07.495652 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:07.496067 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:07.497021 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:07.497029 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:07.497045 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:07.497065 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:07.497130 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:07.497155 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:07.497410 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:07.497545 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 07:08:07.497720 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":337,"threads_started":1}
I20260812 07:08:07.499590 20052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede: Bootstrap starting.
I20260812 07:08:07.502075 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:08:07.502645 20052 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:07.503806 20052 log.cc:824] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:07.506775 20052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede: No bootstrap required, opened a new log
I20260812 07:08:07.509037 20059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:07.506859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44177 (local address 127.11.192.254:40519)
0812 07:08:07.507005 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:07.507011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:07.507389 (+ 378us) server_negotiation.cc:407] Connection header received
0812 07:08:07.507497 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:07.507501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:07.507554 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:07.507653 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:07.508467 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:07.508473 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:07.508477 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:08:07.508499 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:07.508547 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:08:07.508566 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:07.508683 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:07.508728 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 07:08:07.508775 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:08:07.514482 20052 raft_consensus.cc:348] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918ad01804b6408db91adc8af89a7ede" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40519 } }
I20260812 07:08:07.514767 20052 raft_consensus.cc:374] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:08:07.514843 20052 raft_consensus.cc:729] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 918ad01804b6408db91adc8af89a7ede, State: Initialized, Role: FOLLOWER
I20260812 07:08:07.515292 20052 consensus_queue.cc:260] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918ad01804b6408db91adc8af89a7ede" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40519 } }
I20260812 07:08:07.515491 20052 raft_consensus.cc:388] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:08:07.515583 20052 raft_consensus.cc:482] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:08:07.515693 20052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:07.517630 20052 raft_consensus.cc:504] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918ad01804b6408db91adc8af89a7ede" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40519 } }
I20260812 07:08:07.518049 20052 leader_election.cc:302] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 918ad01804b6408db91adc8af89a7ede; no voters:
I20260812 07:08:07.518508 20052 leader_election.cc:288] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:08:07.518623 20068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:07.519217 20068 raft_consensus.cc:686] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede [term 1 LEADER]: Becoming Leader. State: Replica: 918ad01804b6408db91adc8af89a7ede, State: Running, Role: LEADER
I20260812 07:08:07.519757 20068 consensus_queue.cc:237] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918ad01804b6408db91adc8af89a7ede" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40519 } }
I20260812 07:08:07.520457 20052 sys_catalog.cc:564] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:07.522771 20070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede [sys.catalog]: SysCatalogTable state changed. Reason: New leader 918ad01804b6408db91adc8af89a7ede. Latest consensus state: current_term: 1 leader_uuid: "918ad01804b6408db91adc8af89a7ede" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918ad01804b6408db91adc8af89a7ede" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40519 } } }
I20260812 07:08:07.523178 20070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:07.524847 20069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "918ad01804b6408db91adc8af89a7ede" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918ad01804b6408db91adc8af89a7ede" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 40519 } } }
I20260812 07:08:07.525156 20069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:07.525537 20081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:07.530869 20081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:07.539355 20081 catalog_manager.cc:1347] Generated new cluster ID: 4ddb6cbc76fa4ff2b889277c38ab9e02
I20260812 07:08:07.539449 20081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:07.581598 20081 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:07.582113 20081 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:07.591310 20081 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 918ad01804b6408db91adc8af89a7ede: Generated new TSK 0
I20260812 07:08:07.592227 20081 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:08:07.598694 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 19929
W20260812 07:08:07.600108 20043 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38291
W20260812 07:08:07.600736 12035 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 19929: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:08:07.600891 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 19929
2026-08-12T07:08:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2044 ms)
[----------] 18 tests from AuthTokenIssuingTest (37359 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:08:07 dist-test-slave-lgr5 krb5kdc[20098](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:08:07 dist-test-slave-lgr5 krb5kdc[20098](info): set up 2 sockets
Aug 12 07:08:07 dist-test-slave-lgr5 krb5kdc[20098](info): commencing operation
krb5kdc: starting...
W20260812 07:08:08.910097 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.003s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:08:09 dist-test-slave-lgr5 krb5kdc[20098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:08:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:08:09.193424 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:46595
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:42027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:46595,127.11.192.253:36785,127.11.192.252:42413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:09.514750 20121 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:08:09.515070 20121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:09.515169 20121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:09.515285 20121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:09.522235 20121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:09.522321 20121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:09.522419 20121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:09.522486 20121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:09.522579 20121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:09.531785 20121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:42027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:46595,127.11.192.253:36785,127.11.192.252:42413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:46595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:08:09.532914 20121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:09.534096 20121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:09.544826 20140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:09.544920 20137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:09.544901 20142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:09.545672 20121 server_base.cc:1034] running on GCE node
I20260812 07:08:09.546355 20121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:09.547561 20121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:09.548776 20121 hybrid_clock.cc:648] HybridClock initialized: now 1786518489548715 us; error 35 us; skew 500 ppm
I20260812 07:08:09.549096 20121 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:08:09 dist-test-slave-lgr5 krb5kdc[20098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518489, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:09.552009 20121 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:08:09.553285 20121 webserver.cc:460] Webserver started at http://127.11.192.254:44197/ using document root <none> and password file <none>
I20260812 07:08:09.553862 20121 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:09.553952 20121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:09.554203 20121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:09.556882 20121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "0b95a10b37814db2832a4e75abdb109b"
format_stamp: "Formatted at 2026-08-12 07:08:09 on dist-test-slave-lgr5"
server_key: "ff6eff3f7eb081eed8a36b6b6e50125b"
server_key_iv: "f0d5848b65aa0c9cd1b188ec7a6ade13"
server_key_version: "encryptionkey@0"
I20260812 07:08:09.557610 20121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "0b95a10b37814db2832a4e75abdb109b"
format_stamp: "Formatted at 2026-08-12 07:08:09 on dist-test-slave-lgr5"
server_key: "ff6eff3f7eb081eed8a36b6b6e50125b"
server_key_iv: "f0d5848b65aa0c9cd1b188ec7a6ade13"
server_key_version: "encryptionkey@0"
I20260812 07:08:09.562197 20121 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:08:09.565332 20158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:09.566501 20121 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:08:09.566632 20121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "0b95a10b37814db2832a4e75abdb109b"
format_stamp: "Formatted at 2026-08-12 07:08:09 on dist-test-slave-lgr5"
server_key: "ff6eff3f7eb081eed8a36b6b6e50125b"
server_key_iv: "f0d5848b65aa0c9cd1b188ec7a6ade13"
server_key_version: "encryptionkey@0"
I20260812 07:08:09.566793 20121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:09.578505 20121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:09.581888 20121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:09.582093 20121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:09.599082 20121 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:46595
I20260812 07:08:09.599115 20243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:46595 every 8 connection(s)
I20260812 07:08:09.600262 20121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:09.603679 20245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:09.607864 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 20121
I20260812 07:08:09.608112 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:08:09.608942 12035 external_mini_cluster.cc:1428] Setting key d544d515549aabc4f2894141447a3871
I20260812 07:08:09.610810 20245 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46595 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:08:09 dist-test-slave-lgr5 krb5kdc[20098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518489, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:08:09.631335 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:09.614162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53870 (local address 127.11.192.254:46595)
0812 07:08:09.615677 (+ 1515us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:09.615685 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:09.615735 (+ 50us) server_negotiation.cc:407] Connection header received
0812 07:08:09.617216 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:09.617221 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:09.617430 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:09.617600 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:09.618792 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:09.619936 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:09.621267 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:09.621776 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:09.621931 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:09.625057 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:09.625064 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:09.625074 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:09.625102 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:09.627547 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:09.628164 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:09.628172 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:09.628174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:09.628304 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:09.628671 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:09.628674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:09.628677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:09.629115 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:09.629270 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:09.629806 (+ 536us) server_negotiation.cc:299] Negotiation successful
0812 07:08:09.630050 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1315,"thread_start_us":1110,"threads_started":1}
I20260812 07:08:09.631371 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:09.614161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:46595 (local address 127.0.0.1:53870)
0812 07:08:09.614924 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:09.614967 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:09.616891 (+ 1924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:09.617640 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:09.617647 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:09.618277 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:09.618602 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:09.618618 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:09.620114 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:09.620119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:09.621140 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:09.621152 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:09.621989 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:09.621997 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:09.622216 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:09.622798 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:09.622819 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:09.624852 (+ 2033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:09.627658 (+ 2806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:09.627669 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:09.627679 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:09.628031 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:09.628406 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:09.628411 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:09.628415 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:09.628564 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:09.629245 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:09.629252 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:09.629442 (+ 190us) client_negotiation.cc:769] Sending connection context
0812 07:08:09.629644 (+ 202us) client_negotiation.cc:240] Negotiation successful
0812 07:08:09.630189 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"mutex_wait_us":184,"thread_start_us":306,"threads_started":1}
I20260812 07:08:09.633972 20245 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:08:09.635069 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:09.634695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:36785 (local address 127.0.0.1:34020)
0812 07:08:09.634841 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:09.634940 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 07:08:09.635315 20162 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.192.253:36785: connect: Connection refused (error 111)
I20260812 07:08:09.636741 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:09.636405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:36785 (local address 127.0.0.1:34034)
0812 07:08:09.636560 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:09.636622 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 07:08:09.636915 20245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.253:36785: Network error: Client connection negotiation failed: client connection to 127.11.192.253:36785: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.11.192.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.253.keytab.
Entry for principal kudu/127.11.192.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.253.keytab.
I20260812 07:08:09.671788 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.253:36785
--webserver_interface=127.11.192.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.253.keytab
--principal=kudu/127.11.192.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:42027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:46595,127.11.192.253:36785,127.11.192.252:42413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:08:09.686218 20245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } attempt: 1
I20260812 07:08:09.687366 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:09.686924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:36785 (local address 127.0.0.1:34042)
0812 07:08:09.687133 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:09.687242 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 07:08:09.688361 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:09.688016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:36785 (local address 127.0.0.1:34044)
0812 07:08:09.688161 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:09.688232 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 07:08:09.688561 20245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.253:36785: Network error: Client connection negotiation failed: client connection to 127.11.192.253:36785: connect: Connection refused (error 111)
I20260812 07:08:09.756835 20245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } attempt: 2
I20260812 07:08:09.757987 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:09.757515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:36785 (local address 127.0.0.1:34052)
0812 07:08:09.757748 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:09.757835 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 07:08:09.758914 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:09.758575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:36785 (local address 127.0.0.1:34054)
0812 07:08:09.758725 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:09.758792 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 07:08:09.759094 20245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.253:36785: Network error: Client connection negotiation failed: client connection to 127.11.192.253:36785: connect: Connection refused (error 111)
I20260812 07:08:09.850344 20245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } attempt: 3
I20260812 07:08:09.851394 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:09.850950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:36785 (local address 127.0.0.1:34068)
0812 07:08:09.851163 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:09.851257 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 07:08:09.852401 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:09.852016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:36785 (local address 127.0.0.1:34080)
0812 07:08:09.852194 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:09.852267 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 07:08:09.852602 20245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.253:36785: Network error: Client connection negotiation failed: client connection to 127.11.192.253:36785: connect: Connection refused (error 111)
I20260812 07:08:09.995831 20245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } attempt: 4
I20260812 07:08:09.997028 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:09.996483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:36785 (local address 127.0.0.1:34096)
0812 07:08:09.996794 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:09.996881 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260812 07:08:09.997953 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:09.997617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:36785 (local address 127.0.0.1:34110)
0812 07:08:09.997759 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:09.997839 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:36785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 07:08:09.998122 20245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.253:36785: Network error: Client connection negotiation failed: client connection to 127.11.192.253:36785: connect: Connection refused (error 111)
W20260812 07:08:10.007141 20253 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:08:10.007490 20253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:10.007599 20253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:10.007730 20253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:10.014982 20253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:10.015095 20253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:10.015202 20253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:10.015280 20253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:10.015383 20253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:10.025024 20253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:42027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:46595,127.11.192.253:36785,127.11.192.252:42413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.253.keytab
--principal=kudu/127.11.192.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.253:36785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.192.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:08:10.026268 20253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:10.027565 20253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:10.039166 20263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:10.039171 20266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:10.039525 20269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:10.039867 20253 server_base.cc:1034] running on GCE node
I20260812 07:08:10.040660 20253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:10.041947 20253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:10.043166 20253 hybrid_clock.cc:648] HybridClock initialized: now 1786518490043095 us; error 45 us; skew 500 ppm
I20260812 07:08:10.043493 20253 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:08:10 dist-test-slave-lgr5 krb5kdc[20098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518490, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:10.046625 20253 init.cc:377] Logged in from keytab as kudu/127.11.192.253@KRBTEST.COM (short username kudu)
I20260812 07:08:10.047986 20253 webserver.cc:460] Webserver started at http://127.11.192.253:46489/ using document root <none> and password file <none>
I20260812 07:08:10.048671 20253 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:10.048782 20253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:10.049105 20253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:10.052037 20253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-1/data/instance:
uuid: "92d26777f569488ab9f90d0331e4d223"
format_stamp: "Formatted at 2026-08-12 07:08:10 on dist-test-slave-lgr5"
server_key: "ed1d8414d518450a36b09fec88baab5f"
server_key_iv: "a07edb6cdfaab0eed3d4d9626a41fe6e"
server_key_version: "encryptionkey@0"
I20260812 07:08:10.052904 20253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-1/wal/instance:
uuid: "92d26777f569488ab9f90d0331e4d223"
format_stamp: "Formatted at 2026-08-12 07:08:10 on dist-test-slave-lgr5"
server_key: "ed1d8414d518450a36b09fec88baab5f"
server_key_iv: "a07edb6cdfaab0eed3d4d9626a41fe6e"
server_key_version: "encryptionkey@0"
I20260812 07:08:10.057958 20253 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:08:10.061214 20280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:10.062389 20253 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:08:10.062547 20253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-1/wal
uuid: "92d26777f569488ab9f90d0331e4d223"
format_stamp: "Formatted at 2026-08-12 07:08:10 on dist-test-slave-lgr5"
server_key: "ed1d8414d518450a36b09fec88baab5f"
server_key_iv: "a07edb6cdfaab0eed3d4d9626a41fe6e"
server_key_version: "encryptionkey@0"
I20260812 07:08:10.062741 20253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:10.081921 20253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:10.085785 20253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:10.086018 20253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:10.102847 20253 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.253:36785
I20260812 07:08:10.102861 20363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.253:36785 every 8 connection(s)
I20260812 07:08:10.104027 20253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-1/data/info.pb
I20260812 07:08:10.107574 20364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:10.107645 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 20253
I20260812 07:08:10.107833 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-1/wal/instance
I20260812 07:08:10.108410 12035 external_mini_cluster.cc:1428] Setting key c737ae3eff326f201c9ab5c6a2908175
I20260812 07:08:10.116518 20364 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46595 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:08:10 dist-test-slave-lgr5 krb5kdc[20098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518490, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.253@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:08:10.137153 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.119507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53872 (local address 127.11.192.254:46595)
0812 07:08:10.119720 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:10.119726 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:10.120904 (+ 1178us) server_negotiation.cc:407] Connection header received
0812 07:08:10.122698 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:10.122705 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:10.122792 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:10.122919 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:10.124291 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.124966 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.127038 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.127508 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.127620 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.131366 (+ 3746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:10.131372 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:10.131375 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:10.131413 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:10.134193 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.134995 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.134999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.135001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.135137 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.135530 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.135534 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.135536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.135803 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:10.135938 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:10.136616 (+ 678us) server_negotiation.cc:299] Negotiation successful
0812 07:08:10.136846 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:08:10.137993 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.119946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:46595 (local address 127.0.0.1:53872)
0812 07:08:10.120731 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.120773 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:10.122425 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:10.123076 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:10.123083 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:10.123564 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:10.124118 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:10.124143 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.125137 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.125143 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:10.126869 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:10.126886 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.127733 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.127739 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:10.127994 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.128690 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:10.128724 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:10.131113 (+ 2389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:10.134414 (+ 3301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:10.134423 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:10.134437 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:10.134870 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:10.135241 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:10.135246 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:10.135250 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:10.135417 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:10.135926 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:10.135933 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:10.136264 (+ 331us) client_negotiation.cc:769] Sending connection context
0812 07:08:10.136486 (+ 222us) client_negotiation.cc:240] Negotiation successful
0812 07:08:10.136924 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":335,"threads_started":1}
I20260812 07:08:10.139597 20364 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.192.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.252@KRBTEST.COM" created.
Aug 12 07:08:10 dist-test-slave-lgr5 krb5kdc[20098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518490, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.253@KRBTEST.COM for kudu/127.11.192.253@KRBTEST.COM
I20260812 07:08:10.152556 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.140381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:36785 (local address 127.0.0.1:34116)
0812 07:08:10.140537 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.140555 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:10.140657 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:10.141519 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:10.141522 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:10.141886 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:10.142023 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:10.142031 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.143355 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.143360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:10.144367 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:10.144379 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.145098 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.145104 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:10.145263 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.145722 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:10.145738 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:10.147549 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:10.150328 (+ 2779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:10.150332 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:10.150336 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:10.150663 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:10.151004 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:10.151009 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:10.151013 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:10.151133 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:10.151677 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:10.151681 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:10.151814 (+ 133us) client_negotiation.cc:769] Sending connection context
0812 07:08:10.151932 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 07:08:10.152194 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 07:08:10.152765 20372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.140469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34116 (local address 127.11.192.253:36785)
0812 07:08:10.141047 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:10.141055 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:10.141088 (+ 33us) server_negotiation.cc:407] Connection header received
0812 07:08:10.141145 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:10.141152 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:10.141362 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:10.141507 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:10.142181 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.143213 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.144487 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.144973 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.145079 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.147737 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:10.147747 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:10.147757 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:10.147784 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:10.150215 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.150775 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.150781 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.150783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.150893 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.151227 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.151232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.151235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.151558 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:10.151743 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:10.152074 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 07:08:10.152360 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":257,"threads_started":1}
I20260812 07:08:10.155221 20364 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:08:10.156251 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.155806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:42413 (local address 127.0.0.1:50192)
0812 07:08:10.155958 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.156059 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 07:08:10.156541 20284 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
I20260812 07:08:10.158026 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.157682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:42413 (local address 127.0.0.1:50198)
0812 07:08:10.157854 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.157930 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 07:08:10.158244 20364 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:42413: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.252.keytab.
Entry for principal kudu/127.11.192.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.252.keytab.
I20260812 07:08:10.176425 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.252:42413
--webserver_interface=127.11.192.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.252.keytab
--principal=kudu/127.11.192.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:42027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:46595,127.11.192.253:36785,127.11.192.252:42413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:08:10.207511 20364 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } attempt: 1
I20260812 07:08:10.208606 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.208186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:42413 (local address 127.0.0.1:50204)
0812 07:08:10.208361 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.208459 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 07:08:10.209532 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.209216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:42413 (local address 127.0.0.1:50208)
0812 07:08:10.209356 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.209427 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 07:08:10.209728 20364 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:42413: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
I20260812 07:08:10.277927 20364 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } attempt: 2
I20260812 07:08:10.279033 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.278601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:42413 (local address 127.0.0.1:50216)
0812 07:08:10.278782 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.278873 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 07:08:10.280019 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.279708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:42413 (local address 127.0.0.1:50218)
0812 07:08:10.279839 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.279905 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 07:08:10.280244 20364 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:42413: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
I20260812 07:08:10.297355 20245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } attempt: 5
Aug 12 07:08:10 dist-test-slave-lgr5 krb5kdc[20098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518489, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for kudu/127.11.192.253@KRBTEST.COM
I20260812 07:08:10.311226 20372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.298134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34122 (local address 127.11.192.253:36785)
0812 07:08:10.298275 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:10.298279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:10.298297 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:08:10.298426 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:10.298430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:10.298514 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:10.298595 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:10.299681 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.300413 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.302100 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.302554 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.302642 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.306285 (+ 3643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:10.306290 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:10.306293 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:10.306326 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:10.308800 (+ 2474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.309555 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.309558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.309561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.309680 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.310079 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.310082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.310084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.310335 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:10.310440 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:10.310843 (+ 403us) server_negotiation.cc:299] Negotiation successful
0812 07:08:10.311021 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 07:08:10.311380 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.297919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:36785 (local address 127.0.0.1:34122)
0812 07:08:10.298141 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.298163 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:10.298307 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:10.298663 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:10.298667 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:10.299213 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:10.299500 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:10.299518 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.300565 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.300571 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:10.301955 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:10.301973 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.302699 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.302705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:10.302951 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.303576 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:10.303600 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:10.306104 (+ 2504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:10.308940 (+ 2836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:10.308946 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:10.308950 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:10.309442 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:10.309783 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:10.309787 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:10.309791 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:10.309982 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:10.310448 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:10.310453 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:10.310615 (+ 162us) client_negotiation.cc:769] Sending connection context
0812 07:08:10.310771 (+ 156us) client_negotiation.cc:240] Negotiation successful
0812 07:08:10.311050 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 07:08:10.312397 20245 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:08:10.313331 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.312998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:42413 (local address 127.0.0.1:50234)
0812 07:08:10.313138 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.313208 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 07:08:10.314177 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.313897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:42413 (local address 127.0.0.1:50250)
0812 07:08:10.314002 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.314078 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 07:08:10.314354 20245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:42413: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
I20260812 07:08:10.365553 20245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } attempt: 1
I20260812 07:08:10.366715 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.366239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:42413 (local address 127.0.0.1:50254)
0812 07:08:10.366437 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.366538 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 07:08:10.367750 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.367450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:42413 (local address 127.0.0.1:50264)
0812 07:08:10.367584 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.367652 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 07:08:10.367952 20245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:42413: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
I20260812 07:08:10.371430 20364 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } attempt: 3
I20260812 07:08:10.372406 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.371999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:42413 (local address 127.0.0.1:50268)
0812 07:08:10.372194 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.372274 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 07:08:10.373327 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.372929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:42413 (local address 127.0.0.1:50270)
0812 07:08:10.373140 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.373212 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260812 07:08:10.373531 20364 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:42413: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
I20260812 07:08:10.436149 20245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } attempt: 2
I20260812 07:08:10.437294 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.436844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:42413 (local address 127.0.0.1:50278)
0812 07:08:10.437046 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.437135 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 07:08:10.438166 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.437850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:42413 (local address 127.0.0.1:50292)
0812 07:08:10.437988 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.438059 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 07:08:10.438351 20245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:42413: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
W20260812 07:08:10.515339 20377 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:08:10.515671 20377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:10.515774 20377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:10.515897 20377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 07:08:10.516705 20364 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } attempt: 4
I20260812 07:08:10.517729 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.517274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:42413 (local address 127.0.0.1:50294)
0812 07:08:10.517488 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.517573 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 07:08:10.518676 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.518366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:42413 (local address 127.0.0.1:50298)
0812 07:08:10.518503 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.518579 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 07:08:10.518864 20364 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:42413: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
W20260812 07:08:10.523345 20377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:10.523437 20377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:10.523540 20377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:10.523620 20377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:10.523715 20377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:10.533392 20377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:42027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:46595,127.11.192.253:36785,127.11.192.252:42413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.252.keytab
--principal=kudu/127.11.192.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.252:42413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.192.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:08:10.534564 20377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:10.535817 20377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:08:10.544528 20245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } attempt: 3
I20260812 07:08:10.545456 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.545027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:42413 (local address 127.0.0.1:50300)
0812 07:08:10.545187 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.545282 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 07:08:10.546959 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.546372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:42413 (local address 127.0.0.1:50304)
0812 07:08:10.546629 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.546755 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 07:08:10.547124 20388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:10.547166 20389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:10.547456 20245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:42413: Network error: Client connection negotiation failed: client connection to 127.11.192.252:42413: connect: Connection refused (error 111)
W20260812 07:08:10.547938 20391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:10.548552 20377 server_base.cc:1034] running on GCE node
I20260812 07:08:10.549228 20377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:10.550444 20377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:10.551658 20377 hybrid_clock.cc:648] HybridClock initialized: now 1786518490551594 us; error 35 us; skew 500 ppm
I20260812 07:08:10.551940 20377 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:08:10 dist-test-slave-lgr5 krb5kdc[20098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518490, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:10.559253 20377 init.cc:377] Logged in from keytab as kudu/127.11.192.252@KRBTEST.COM (short username kudu)
I20260812 07:08:10.560623 20377 webserver.cc:460] Webserver started at http://127.11.192.252:38961/ using document root <none> and password file <none>
I20260812 07:08:10.561359 20377 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:10.561445 20377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:10.561717 20377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:10.564459 20377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-2/data/instance:
uuid: "44a040698c9847e590b8a2c4e8a0ce52"
format_stamp: "Formatted at 2026-08-12 07:08:10 on dist-test-slave-lgr5"
server_key: "8fb7f563befcd94bcf2112d9566da1ed"
server_key_iv: "f6c4bafb8ade28bfa50ef96b2b455863"
server_key_version: "encryptionkey@0"
I20260812 07:08:10.565182 20377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-2/wal/instance:
uuid: "44a040698c9847e590b8a2c4e8a0ce52"
format_stamp: "Formatted at 2026-08-12 07:08:10 on dist-test-slave-lgr5"
server_key: "8fb7f563befcd94bcf2112d9566da1ed"
server_key_iv: "f6c4bafb8ade28bfa50ef96b2b455863"
server_key_version: "encryptionkey@0"
I20260812 07:08:10.570314 20377 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260812 07:08:10.573777 20402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:10.574934 20377 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:08:10.575095 20377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-2/wal
uuid: "44a040698c9847e590b8a2c4e8a0ce52"
format_stamp: "Formatted at 2026-08-12 07:08:10 on dist-test-slave-lgr5"
server_key: "8fb7f563befcd94bcf2112d9566da1ed"
server_key_iv: "f6c4bafb8ade28bfa50ef96b2b455863"
server_key_version: "encryptionkey@0"
I20260812 07:08:10.575268 20377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:10.593744 20377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:10.597963 20377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:10.598196 20377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:10.616751 20377 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.252:42413
I20260812 07:08:10.616755 20484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.252:42413 every 8 connection(s)
I20260812 07:08:10.618096 20377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-2/data/info.pb
I20260812 07:08:10.621551 20486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:10.622340 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 20377
I20260812 07:08:10.622519 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518136653111-12035-0/minicluster-data/master-2/wal/instance
I20260812 07:08:10.622996 12035 external_mini_cluster.cc:1428] Setting key a59ddf4994d6f361e50b38f37c478bc7
I20260812 07:08:10.630223 20486 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46595 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:08:10 dist-test-slave-lgr5 krb5kdc[20098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.252@KRBTEST.COM
Aug 12 07:08:10 dist-test-slave-lgr5 krb5kdc[20098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518490, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.252@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:08:10.645740 20490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.624814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50320 (local address 127.11.192.252:42413)
0812 07:08:10.625616 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:10.625627 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:10.625675 (+ 48us) server_negotiation.cc:407] Connection header received
0812 07:08:10.627361 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:10.627383 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:10.627779 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:10.628163 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:10.629883 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.631230 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.632784 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.633874 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.634750 (+ 876us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.638798 (+ 4048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:10.638806 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:10.638816 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:10.638856 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:10.641697 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.642370 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.642378 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.642386 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.642553 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.642960 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.642965 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.642968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.643596 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:10.643890 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:10.644295 (+ 405us) server_negotiation.cc:299] Negotiation successful
0812 07:08:10.644660 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":368,"threads_started":1}
I20260812 07:08:10.647701 20491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.632774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:46595 (local address 127.0.0.1:53882)
0812 07:08:10.633459 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.633490 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:10.633633 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:10.633998 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:10.634006 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:10.634790 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:10.635133 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:10.635144 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.636518 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.636521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:10.637926 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:10.637944 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.638733 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.638740 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:10.638901 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.639517 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:10.639536 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:10.641748 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:10.645211 (+ 3463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:10.645220 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:10.645225 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:10.645650 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:10.646090 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:10.646095 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:10.646099 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:10.646278 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:10.646792 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:10.646809 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:10.646999 (+ 190us) client_negotiation.cc:769] Sending connection context
0812 07:08:10.647164 (+ 165us) client_negotiation.cc:240] Negotiation successful
0812 07:08:10.647395 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":355,"threads_started":1}
I20260812 07:08:10.647838 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.632698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53882 (local address 127.11.192.254:46595)
0812 07:08:10.632872 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:10.632876 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:10.633588 (+ 712us) server_negotiation.cc:407] Connection header received
0812 07:08:10.633743 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:10.633747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:10.633822 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:10.633920 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:10.635287 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.636359 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.638070 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.638492 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.638581 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.641946 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:10.641951 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:10.641954 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:10.641988 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:10.645079 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.645778 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.645783 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.645786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.645954 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.646410 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.646413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.646415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.646667 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:10.646775 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:10.647300 (+ 525us) server_negotiation.cc:299] Negotiation successful
0812 07:08:10.647509 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:08:10.649026 20486 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:08:10 dist-test-slave-lgr5 krb5kdc[20098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518490, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.252@KRBTEST.COM for kudu/127.11.192.253@KRBTEST.COM
Aug 12 07:08:10 dist-test-slave-lgr5 krb5kdc[20098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:08:10.661440 20491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.649656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:36785 (local address 127.0.0.1:34136)
0812 07:08:10.649812 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.649830 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:10.649916 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:10.650203 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:10.650205 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:10.650523 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:10.650656 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:10.650664 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.651584 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.651590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:10.652567 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:10.652581 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.653304 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.653311 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:10.653509 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.654104 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:10.654121 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:10.656164 (+ 2043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:10.659065 (+ 2901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:10.659071 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:10.659074 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:10.659520 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:10.659874 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:10.659878 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:10.659882 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:10.660070 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:10.660624 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:10.660629 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:10.660787 (+ 158us) client_negotiation.cc:769] Sending connection context
0812 07:08:10.660914 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 07:08:10.661127 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 07:08:10.661504 20372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.649750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34136 (local address 127.11.192.253:36785)
0812 07:08:10.649931 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:10.649937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:10.649955 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:08:10.650012 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:10.650016 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:10.650091 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:10.650195 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:10.650811 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.651438 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.652716 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.653164 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.653260 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.656345 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:10.656352 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:10.656357 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:10.656396 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:10.658914 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.659644 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.659647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.659649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.659776 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.660193 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.660198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.660202 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.660510 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:10.660652 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:10.661052 (+ 400us) server_negotiation.cc:299] Negotiation successful
0812 07:08:10.661243 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:08:10.662305 20486 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:08:10.662782 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.648929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53886 (local address 127.11.192.254:46595)
0812 07:08:10.649089 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:10.649095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:10.649191 (+ 96us) server_negotiation.cc:407] Connection header received
0812 07:08:10.649331 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:10.649335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:10.649396 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:10.649501 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:10.650783 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.651545 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.652786 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.653183 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.653311 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.657224 (+ 3913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:10.657228 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:10.657231 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:10.657259 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:10.659902 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.660625 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.660630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.660633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.660779 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.661178 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.661184 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.661188 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.661529 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:10.661769 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:10.662288 (+ 519us) server_negotiation.cc:299] Negotiation successful
0812 07:08:10.662504 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 07:08:10 dist-test-slave-lgr5 krb5kdc[20098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518490, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.252@KRBTEST.COM for kudu/127.11.192.252@KRBTEST.COM
Aug 12 07:08:10 dist-test-slave-lgr5 krb5kdc[20098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.253@KRBTEST.COM
I20260812 07:08:10.674846 20491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.662865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:42413 (local address 127.0.0.1:50336)
0812 07:08:10.663035 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.663053 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:10.663142 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:10.663450 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:10.663453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:10.663841 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:10.663999 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:10.664011 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.665224 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.665228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:10.666169 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:10.666183 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.666817 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.666823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:10.667005 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.667521 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:10.667537 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:10.669467 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:10.672645 (+ 3178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:10.672648 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:10.672651 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:10.672994 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:10.673353 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:10.673357 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:10.673361 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:10.673539 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:10.674013 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:10.674018 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:10.674135 (+ 117us) client_negotiation.cc:769] Sending connection context
0812 07:08:10.674252 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 07:08:10.674497 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 07:08:10.674991 20490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.663070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50336 (local address 127.11.192.252:42413)
0812 07:08:10.663206 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:10.663211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:10.663228 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:08:10.663273 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:10.663277 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:10.663339 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:10.663438 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:10.664151 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.665108 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.666287 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.666673 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.666771 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.669646 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:10.669653 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:10.669657 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:10.669692 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:10.672520 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.673109 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.673114 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.673117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.673247 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.673648 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.673651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.673653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.673896 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:10.674017 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:10.674346 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 07:08:10.674656 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":107,"server-negotiator.queue_time_us":41}
I20260812 07:08:10.678282 20372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.664355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34138 (local address 127.11.192.253:36785)
0812 07:08:10.664498 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:10.664501 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:10.664623 (+ 122us) server_negotiation.cc:407] Connection header received
0812 07:08:10.664756 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:10.664759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:10.664806 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:10.664872 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:10.666052 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.667008 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.668189 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.668691 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.668782 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.673335 (+ 4553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:10.673341 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:10.673345 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:10.673382 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:10.675848 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.676490 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.676495 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.676499 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.676664 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.676981 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.676986 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.676989 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.677287 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:10.677547 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:10.677705 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 07:08:10.677930 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 07:08:10.680440 20486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44a040698c9847e590b8a2c4e8a0ce52: Bootstrap starting.
I20260812 07:08:10.682592 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:08:10.683656 20486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44a040698c9847e590b8a2c4e8a0ce52: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:10.684706 20486 log.cc:824] T 00000000000000000000000000000000 P 44a040698c9847e590b8a2c4e8a0ce52: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:10.687189 20486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44a040698c9847e590b8a2c4e8a0ce52: No bootstrap required, opened a new log
I20260812 07:08:10.685185 20204 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0b95a10b37814db2832a4e75abdb109b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53886
I20260812 07:08:10.694051 20486 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44a040698c9847e590b8a2c4e8a0ce52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b95a10b37814db2832a4e75abdb109b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46595 } } peers { permanent_uuid: "92d26777f569488ab9f90d0331e4d223" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } } peers { permanent_uuid: "44a040698c9847e590b8a2c4e8a0ce52" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } }
I20260812 07:08:10.694326 20486 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44a040698c9847e590b8a2c4e8a0ce52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44a040698c9847e590b8a2c4e8a0ce52, State: Initialized, Role: FOLLOWER
I20260812 07:08:10.694726 20486 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44a040698c9847e590b8a2c4e8a0ce52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b95a10b37814db2832a4e75abdb109b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46595 } } peers { permanent_uuid: "92d26777f569488ab9f90d0331e4d223" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } } peers { permanent_uuid: "44a040698c9847e590b8a2c4e8a0ce52" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } }
I20260812 07:08:10.695607 20494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44a040698c9847e590b8a2c4e8a0ce52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b95a10b37814db2832a4e75abdb109b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46595 } } peers { permanent_uuid: "92d26777f569488ab9f90d0331e4d223" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } } peers { permanent_uuid: "44a040698c9847e590b8a2c4e8a0ce52" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } } }
I20260812 07:08:10.696023 20494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44a040698c9847e590b8a2c4e8a0ce52 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:08:10.696522 20486 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44a040698c9847e590b8a2c4e8a0ce52 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 07:08:10.705404 20512 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 44a040698c9847e590b8a2c4e8a0ce52: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:08:10.705535 20512 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:08:10.703254 20377 master_runner.cc:186] Error getting master registration for 127.11.192.254:46595: OK, instance_id { permanent_uuid: "0b95a10b37814db2832a4e75abdb109b" instance_seqno: 1786518489594409 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 07:08:10.713842 20377 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 07:08:10.724660 20245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } attempt: 4
Aug 12 07:08:10 dist-test-slave-lgr5 krb5kdc[20098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518489, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for kudu/127.11.192.252@KRBTEST.COM
I20260812 07:08:10.740000 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.725228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:42413 (local address 127.0.0.1:50340)
0812 07:08:10.725397 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.725416 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:10.725513 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:10.725835 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:10.725838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:10.726199 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:10.726364 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:10.726373 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.727816 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.727821 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:10.729312 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:10.729329 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.730325 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.730331 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:10.730536 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.731152 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:10.731170 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:10.733540 (+ 2370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:10.737397 (+ 3857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:10.737401 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:10.737404 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:10.737863 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:10.738240 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:10.738244 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:10.738249 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:10.738422 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:10.739056 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:10.739061 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:10.739201 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 07:08:10.739353 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 07:08:10.739614 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 07:08:10.740041 20490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.725368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50340 (local address 127.11.192.252:42413)
0812 07:08:10.725562 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:10.725568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:10.725587 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:08:10.725641 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:10.725646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:10.725720 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:10.725819 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:10.726503 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.727679 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.729456 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.730173 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.730302 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.733716 (+ 3414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:10.733724 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:10.733729 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:10.733765 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:10.737276 (+ 3511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.737992 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.737998 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.738002 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.738141 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.738527 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.738532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.738535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.738936 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:10.739120 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:10.739473 (+ 353us) server_negotiation.cc:299] Negotiation successful
0812 07:08:10.739694 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:08:10.744642 20245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b: Bootstrap starting.
I20260812 07:08:10.747526 20245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:10.748741 20245 log.cc:824] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:10.751293 20245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b: No bootstrap required, opened a new log
I20260812 07:08:10.758425 20245 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b95a10b37814db2832a4e75abdb109b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46595 } } peers { permanent_uuid: "92d26777f569488ab9f90d0331e4d223" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } } peers { permanent_uuid: "44a040698c9847e590b8a2c4e8a0ce52" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } }
I20260812 07:08:10.758818 20245 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b95a10b37814db2832a4e75abdb109b, State: Initialized, Role: FOLLOWER
I20260812 07:08:10.759251 20245 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b95a10b37814db2832a4e75abdb109b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46595 } } peers { permanent_uuid: "92d26777f569488ab9f90d0331e4d223" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } } peers { permanent_uuid: "44a040698c9847e590b8a2c4e8a0ce52" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } }
I20260812 07:08:10.760149 20515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b95a10b37814db2832a4e75abdb109b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46595 } } peers { permanent_uuid: "92d26777f569488ab9f90d0331e4d223" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } } peers { permanent_uuid: "44a040698c9847e590b8a2c4e8a0ce52" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } } }
I20260812 07:08:10.760586 20515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:08:10.761224 20245 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:10.761351 20204 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 07:08:10.761566 20204 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:10.764895 20204 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b95a10b37814db2832a4e75abdb109b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46595 } } peers { permanent_uuid: "92d26777f569488ab9f90d0331e4d223" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } } peers { permanent_uuid: "44a040698c9847e590b8a2c4e8a0ce52" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } }
I20260812 07:08:10.766325 20204 leader_election.cc:288] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b [CANDIDATE]: Term 1 election: Requested vote from peers 92d26777f569488ab9f90d0331e4d223 (127.11.192.253:36785), 44a040698c9847e590b8a2c4e8a0ce52 (127.11.192.252:42413)
I20260812 07:08:10.767578 20450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0b95a10b37814db2832a4e75abdb109b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "44a040698c9847e590b8a2c4e8a0ce52"
I20260812 07:08:10.768095 20450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44a040698c9847e590b8a2c4e8a0ce52 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:10.771647 20450 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 44a040698c9847e590b8a2c4e8a0ce52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b95a10b37814db2832a4e75abdb109b in term 1.
I20260812 07:08:10.772367 20162 leader_election.cc:302] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b95a10b37814db2832a4e75abdb109b, 44a040698c9847e590b8a2c4e8a0ce52; no voters:
I20260812 07:08:10.772842 20515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:10.778028 20515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b [term 1 LEADER]: Becoming Leader. State: Replica: 0b95a10b37814db2832a4e75abdb109b, State: Running, Role: LEADER
I20260812 07:08:10.778715 20515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b95a10b37814db2832a4e75abdb109b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46595 } } peers { permanent_uuid: "92d26777f569488ab9f90d0331e4d223" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } } peers { permanent_uuid: "44a040698c9847e590b8a2c4e8a0ce52" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } }
I20260812 07:08:10.791275 20523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b95a10b37814db2832a4e75abdb109b. Latest consensus state: current_term: 1 leader_uuid: "0b95a10b37814db2832a4e75abdb109b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b95a10b37814db2832a4e75abdb109b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46595 } } peers { permanent_uuid: "92d26777f569488ab9f90d0331e4d223" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } } peers { permanent_uuid: "44a040698c9847e590b8a2c4e8a0ce52" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } } }
I20260812 07:08:10.791675 20523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:10.818053 20364 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } attempt: 5
I20260812 07:08:10.818941 20532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:10.831348 20532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:10.837114 20372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.802863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34140 (local address 127.11.192.253:36785)
0812 07:08:10.803041 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:10.803046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:10.810300 (+ 7254us) server_negotiation.cc:407] Connection header received
0812 07:08:10.811315 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:10.811322 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:10.811418 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:10.811517 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:10.813747 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.814986 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.817033 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.817713 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.817815 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.828933 (+ 11118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:10.828938 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:10.828941 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:10.828973 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:10.833940 (+ 4967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.834731 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.834737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.834741 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.834916 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.835364 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.835369 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.835373 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.835729 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:10.835949 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:10.836529 (+ 580us) server_negotiation.cc:299] Negotiation successful
0812 07:08:10.836749 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260812 07:08:10.837477 20169 tablet.cc:2369] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 07:08:10 dist-test-slave-lgr5 krb5kdc[20098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518490, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.253@KRBTEST.COM for kudu/127.11.192.252@KRBTEST.COM
W20260812 07:08:10.850265 20411 tablet.cc:2369] T 00000000000000000000000000000000 P 44a040698c9847e590b8a2c4e8a0ce52: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:08:10.855005 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.802537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53892 (local address 127.11.192.254:46595)
0812 07:08:10.802713 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:10.802719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:10.811851 (+ 9132us) server_negotiation.cc:407] Connection header received
0812 07:08:10.812359 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:10.812366 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:10.812443 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:10.812521 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:10.821457 (+ 8936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.822557 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.832499 (+ 9942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.833268 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.833366 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.841356 (+ 7990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:10.841364 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:10.841370 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:10.841408 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:10.844891 (+ 3483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.851053 (+ 6162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.851059 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.851062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.851232 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.852183 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.852188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.852192 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.852523 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:10.852759 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:10.854403 (+ 1644us) server_negotiation.cc:299] Negotiation successful
0812 07:08:10.854640 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 07:08:10.855660 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.818565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:42413 (local address 127.0.0.1:50354)
0812 07:08:10.818816 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:10.818832 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:10.818929 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:10.829663 (+ 10734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:10.829668 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:10.830139 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:10.830332 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:10.830345 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.841721 (+ 11376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.841726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:10.843069 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:10.843083 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.843890 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.843896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:10.844150 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.844698 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:10.844720 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:10.848022 (+ 3302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:10.852292 (+ 4270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:10.852297 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:10.852301 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:10.852796 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:10.853266 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:10.853271 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:10.853274 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:10.853439 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:10.854143 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:10.854148 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:10.854302 (+ 154us) client_negotiation.cc:769] Sending connection context
0812 07:08:10.854455 (+ 153us) client_negotiation.cc:240] Negotiation successful
0812 07:08:10.854735 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141}
I20260812 07:08:10.856423 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.819318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50354 (local address 127.11.192.252:42413)
0812 07:08:10.829363 (+ 10045us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:10.829369 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:10.829389 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:08:10.829437 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:10.829441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:10.829521 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:10.829628 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:10.840175 (+ 10547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.841549 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.843238 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.843746 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.843854 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.848297 (+ 4443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:10.848305 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:10.848310 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:10.848347 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:10.852112 (+ 3765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.852936 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.852942 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.852945 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.853127 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.853582 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.853587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.853590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.853931 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:10.854066 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:10.855426 (+ 1360us) server_negotiation.cc:299] Negotiation successful
0812 07:08:10.855958 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":9910,"thread_start_us":9351,"threads_started":1}
I20260812 07:08:10.861334 20364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92d26777f569488ab9f90d0331e4d223: Bootstrap starting.
I20260812 07:08:10.832784 20121 master_runner.cc:186] Error getting master registration for 127.11.192.253:36785: OK, instance_id { permanent_uuid: "92d26777f569488ab9f90d0331e4d223" instance_seqno: 1786518490098140 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 07:08:10.863426 20121 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 07:08:10.865376 20364 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92d26777f569488ab9f90d0331e4d223: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:10.866789 20364 log.cc:824] T 00000000000000000000000000000000 P 92d26777f569488ab9f90d0331e4d223: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:10.866878 20490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:10.815012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50342 (local address 127.11.192.252:42413)
0812 07:08:10.815200 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:10.815205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:10.825006 (+ 9801us) server_negotiation.cc:407] Connection header received
0812 07:08:10.825371 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:10.825377 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:10.825454 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:10.825530 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:10.840230 (+ 14700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.841425 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.843231 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:10.844725 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:10.844854 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:10.852491 (+ 7637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:10.852498 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:10.852503 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:10.852539 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:10.856629 (+ 4090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.862573 (+ 5944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.862583 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.862586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.862807 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:10.863731 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:10.863765 (+ 34us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:10.863768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:10.864326 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:10.864778 (+ 452us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:10.865780 (+ 1002us) server_negotiation.cc:299] Negotiation successful
0812 07:08:10.866190 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":784,"server-negotiator.queue_time_us":80}
I20260812 07:08:10.878870 20450 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 44a040698c9847e590b8a2c4e8a0ce52 [term 1 FOLLOWER]: Refusing update from remote peer 0b95a10b37814db2832a4e75abdb109b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:10.879993 20364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92d26777f569488ab9f90d0331e4d223: No bootstrap required, opened a new log
I20260812 07:08:10.880281 20515 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b [LEADER]: Connected to new peer: Peer: permanent_uuid: "44a040698c9847e590b8a2c4e8a0ce52" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:10.890352 20364 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92d26777f569488ab9f90d0331e4d223 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b95a10b37814db2832a4e75abdb109b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46595 } } peers { permanent_uuid: "92d26777f569488ab9f90d0331e4d223" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } } peers { permanent_uuid: "44a040698c9847e590b8a2c4e8a0ce52" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } }
I20260812 07:08:10.890733 20364 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92d26777f569488ab9f90d0331e4d223 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d26777f569488ab9f90d0331e4d223, State: Initialized, Role: FOLLOWER
I20260812 07:08:10.891203 20364 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92d26777f569488ab9f90d0331e4d223 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b95a10b37814db2832a4e75abdb109b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46595 } } peers { permanent_uuid: "92d26777f569488ab9f90d0331e4d223" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } } peers { permanent_uuid: "44a040698c9847e590b8a2c4e8a0ce52" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } }
I20260812 07:08:10.892562 20564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92d26777f569488ab9f90d0331e4d223 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b95a10b37814db2832a4e75abdb109b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46595 } } peers { permanent_uuid: "92d26777f569488ab9f90d0331e4d223" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } } peers { permanent_uuid: "44a040698c9847e590b8a2c4e8a0ce52" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } } }
I20260812 07:08:10.893311 20564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92d26777f569488ab9f90d0331e4d223 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:08:10.893580 20364 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92d26777f569488ab9f90d0331e4d223 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:10.893666 20320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0b95a10b37814db2832a4e75abdb109b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "92d26777f569488ab9f90d0331e4d223"
I20260812 07:08:10.893953 20319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92d26777f569488ab9f90d0331e4d223 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:10.897709 20319 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 92d26777f569488ab9f90d0331e4d223 [term 1 FOLLOWER]: Refusing update from remote peer 0b95a10b37814db2832a4e75abdb109b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:10.901316 20523 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b [LEADER]: Connected to new peer: Peer: permanent_uuid: "92d26777f569488ab9f90d0331e4d223" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:10.903185 20320 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 92d26777f569488ab9f90d0331e4d223 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b95a10b37814db2832a4e75abdb109b in term 1.
I20260812 07:08:10.903903 20494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44a040698c9847e590b8a2c4e8a0ce52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b95a10b37814db2832a4e75abdb109b. Latest consensus state: current_term: 1 leader_uuid: "0b95a10b37814db2832a4e75abdb109b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b95a10b37814db2832a4e75abdb109b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46595 } } peers { permanent_uuid: "92d26777f569488ab9f90d0331e4d223" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } } peers { permanent_uuid: "44a040698c9847e590b8a2c4e8a0ce52" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } } }
I20260812 07:08:10.904245 20494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44a040698c9847e590b8a2c4e8a0ce52 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:08:10.907168 20556 mvcc.cc:205] Tried to move back new op lower bound from 7317579738626580480 to 7317579738236010496. Current Snapshot: MvccSnapshot[applied={T|T < 7317579738626580480}]
I20260812 07:08:10.918566 20515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b95a10b37814db2832a4e75abdb109b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b95a10b37814db2832a4e75abdb109b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46595 } } peers { permanent_uuid: "92d26777f569488ab9f90d0331e4d223" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } } peers { permanent_uuid: "44a040698c9847e590b8a2c4e8a0ce52" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } } }
I20260812 07:08:10.918841 20515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:10.922837 20532 catalog_manager.cc:1347] Generated new cluster ID: 75f514325c4a42af8ca86e28827823c9
I20260812 07:08:10.922930 20532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:10.924176 20494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44a040698c9847e590b8a2c4e8a0ce52 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b95a10b37814db2832a4e75abdb109b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b95a10b37814db2832a4e75abdb109b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46595 } } peers { permanent_uuid: "92d26777f569488ab9f90d0331e4d223" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } } peers { permanent_uuid: "44a040698c9847e590b8a2c4e8a0ce52" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } } }
I20260812 07:08:10.924401 20494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44a040698c9847e590b8a2c4e8a0ce52 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:08:10.933379 20564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92d26777f569488ab9f90d0331e4d223 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b95a10b37814db2832a4e75abdb109b. Latest consensus state: current_term: 1 leader_uuid: "0b95a10b37814db2832a4e75abdb109b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b95a10b37814db2832a4e75abdb109b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46595 } } peers { permanent_uuid: "92d26777f569488ab9f90d0331e4d223" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } } peers { permanent_uuid: "44a040698c9847e590b8a2c4e8a0ce52" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } } }
I20260812 07:08:10.933682 20564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92d26777f569488ab9f90d0331e4d223 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:08:10.934849 20523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b95a10b37814db2832a4e75abdb109b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b95a10b37814db2832a4e75abdb109b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46595 } } peers { permanent_uuid: "92d26777f569488ab9f90d0331e4d223" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } } peers { permanent_uuid: "44a040698c9847e590b8a2c4e8a0ce52" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } } }
I20260812 07:08:10.935143 20523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:10.936679 20568 mvcc.cc:205] Tried to move back new op lower bound from 7317579738626580480 to 7317579738236010496. Current Snapshot: MvccSnapshot[applied={T|T < 7317579738626580480}]
I20260812 07:08:10.952589 20564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92d26777f569488ab9f90d0331e4d223 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b95a10b37814db2832a4e75abdb109b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b95a10b37814db2832a4e75abdb109b" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 46595 } } peers { permanent_uuid: "92d26777f569488ab9f90d0331e4d223" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 36785 } } peers { permanent_uuid: "44a040698c9847e590b8a2c4e8a0ce52" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 42413 } } }
I20260812 07:08:10.952832 20564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92d26777f569488ab9f90d0331e4d223 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:08:10.985922 20532 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:10.987654 20532 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:11.001389 20588 catalog_manager.cc:1259] Loaded cluster ID: 75f514325c4a42af8ca86e28827823c9
I20260812 07:08:11.001569 20588 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 92d26777f569488ab9f90d0331e4d223: loading cluster ID for follower catalog manager: success
I20260812 07:08:11.005195 20588 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 92d26777f569488ab9f90d0331e4d223: acquiring CA information for follower catalog manager: success
W20260812 07:08:11.006516 20588 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260812 07:08:11.010753 20532 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b95a10b37814db2832a4e75abdb109b: Generated new TSK 0
I20260812 07:08:11.011683 20532 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 07:08:11.139500 20372 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:11.124041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34146 (local address 127.11.192.253:36785)
0812 07:08:11.124290 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:11.124296 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:11.135781 (+ 11485us) server_negotiation.cc:407] Connection header received
0812 07:08:11.137923 (+ 2142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:11.137931 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:11.138017 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:11.138096 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:11.139257 (+ 1161us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34146: BlockingRecv error: recv got EOF from 127.0.0.1:34146 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
W20260812 07:08:11.143674 20490 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:11.136357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50368 (local address 127.11.192.252:42413)
0812 07:08:11.136543 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:11.136548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:11.141479 (+ 4931us) server_negotiation.cc:407] Connection header received
0812 07:08:11.142131 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:11.142136 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:11.142222 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:11.142301 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:11.143383 (+ 1082us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50368: BlockingRecv error: recv got EOF from 127.0.0.1:50368 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260812 07:08:11.149307 20249 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:11.132255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53908 (local address 127.11.192.254:46595)
0812 07:08:11.132474 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:11.132480 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:11.140733 (+ 8253us) server_negotiation.cc:407] Connection header received
0812 07:08:11.140888 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:11.140894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:11.140978 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:11.141103 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:11.149076 (+ 7973us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53908: BlockingRecv error: recv got EOF from 127.0.0.1:53908 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260812 07:08:11.197647 20372 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:11.183644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34148 (local address 127.11.192.253:36785)
0812 07:08:11.183847 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:11.183853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:11.195439 (+ 11586us) server_negotiation.cc:407] Connection header received
0812 07:08:11.195811 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:11.195818 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:11.195970 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:11.196060 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:11.197287 (+ 1227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34148: BlockingRecv error: recv got EOF from 127.0.0.1:34148 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260812 07:08:11.245930 20372 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:11.233115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34164 (local address 127.11.192.253:36785)
0812 07:08:11.233332 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:11.233339 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:11.243859 (+ 10520us) server_negotiation.cc:407] Connection header received
0812 07:08:11.244417 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:11.244424 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:11.244530 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:11.244619 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:11.245674 (+ 1055us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34164: BlockingRecv error: recv got EOF from 127.0.0.1:34164 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260812 07:08:11.282270 20372 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:11.277507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34168 (local address 127.11.192.253:36785)
0812 07:08:11.277700 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:11.277707 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:11.280671 (+ 2964us) server_negotiation.cc:407] Connection header received
0812 07:08:11.280880 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:11.280885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:11.281011 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:11.281129 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:11.282004 (+ 875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34168: BlockingRecv error: recv got EOF from 127.0.0.1:34168 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260812 07:08:11.291823 20372 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:11.289531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34178 (local address 127.11.192.253:36785)
0812 07:08:11.289750 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:11.289757 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:11.290306 (+ 549us) server_negotiation.cc:407] Connection header received
0812 07:08:11.290516 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:11.290522 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:11.290612 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:11.290722 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:11.291574 (+ 852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34178: BlockingRecv error: recv got EOF from 127.0.0.1:34178 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
W20260812 07:08:11.305351 20372 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:11.303181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34184 (local address 127.11.192.253:36785)
0812 07:08:11.303382 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:11.303389 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:11.303871 (+ 482us) server_negotiation.cc:407] Connection header received
0812 07:08:11.304059 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:11.304064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:11.304182 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:11.304284 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:11.305125 (+ 841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34184: BlockingRecv error: recv got EOF from 127.0.0.1:34184 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260812 07:08:11.327113 20372 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:11.324931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34188 (local address 127.11.192.253:36785)
0812 07:08:11.325103 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:11.325109 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:11.325609 (+ 500us) server_negotiation.cc:407] Connection header received
0812 07:08:11.325803 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:11.325809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:11.325897 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:11.326031 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:11.326858 (+ 827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34188: BlockingRecv error: recv got EOF from 127.0.0.1:34188 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260812 07:08:11.366564 20372 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:11.364209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34202 (local address 127.11.192.253:36785)
0812 07:08:11.364410 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:11.364416 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:11.364959 (+ 543us) server_negotiation.cc:407] Connection header received
0812 07:08:11.365187 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:11.365193 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:11.365292 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:11.365428 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:11.366265 (+ 837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34202: BlockingRecv error: recv got EOF from 127.0.0.1:34202 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260812 07:08:11.436242 20372 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:11.434230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34214 (local address 127.11.192.253:36785)
0812 07:08:11.434453 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:11.434460 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:11.434812 (+ 352us) server_negotiation.cc:407] Connection header received
0812 07:08:11.434979 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:11.434984 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:11.435090 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:11.435197 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:11.435992 (+ 795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34214: BlockingRecv error: recv got EOF from 127.0.0.1:34214 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260812 07:08:11.569523 20372 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:11.567696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34224 (local address 127.11.192.253:36785)
0812 07:08:11.567857 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:11.567861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:11.568341 (+ 480us) server_negotiation.cc:407] Connection header received
0812 07:08:11.568493 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:11.568497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:11.568574 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:11.568654 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:11.569299 (+ 645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34224: BlockingRecv error: recv got EOF from 127.0.0.1:34224 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:08:11.707465 20512 catalog_manager.cc:1259] Loaded cluster ID: 75f514325c4a42af8ca86e28827823c9
I20260812 07:08:11.707576 20512 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 44a040698c9847e590b8a2c4e8a0ce52: loading cluster ID for follower catalog manager: success
I20260812 07:08:11.710033 20512 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 44a040698c9847e590b8a2c4e8a0ce52: acquiring CA information for follower catalog manager: success
I20260812 07:08:11.711759 20512 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 44a040698c9847e590b8a2c4e8a0ce52: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 07:08:11.830549 20372 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:11.828982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34240 (local address 127.11.192.253:36785)
0812 07:08:11.829151 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:11.829155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:11.829421 (+ 266us) server_negotiation.cc:407] Connection header received
0812 07:08:11.829551 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:11.829555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:11.829638 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:11.829727 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:11.830386 (+ 659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34240: BlockingRecv error: recv got EOF from 127.0.0.1:34240 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:08:12.009260 20588 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 92d26777f569488ab9f90d0331e4d223: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 07:08:12.351522 20700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:12.346269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34254 (local address 127.11.192.253:36785)
0812 07:08:12.346935 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:12.346940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:12.346957 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:08:12.347022 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:12.347025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:12.347180 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:12.347290 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:12.347650 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:12.348389 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:12.349603 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:12.349984 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:12.350076 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:12.350499 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:12.351009 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:12.351122 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:12.351159 (+ 37us) server_negotiation.cc:299] Negotiation successful
0812 07:08:12.351223 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":455,"threads_started":1}
I20260812 07:08:12.362175 20709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:12.357156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50384 (local address 127.11.192.252:42413)
0812 07:08:12.357551 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:12.357555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:12.357647 (+ 92us) server_negotiation.cc:407] Connection header received
0812 07:08:12.357825 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:12.357829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:12.357967 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:12.358094 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:12.358380 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:12.359049 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:12.360284 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:12.360687 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:12.360772 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:12.361162 (+ 390us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:12.361668 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:12.361791 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:12.361875 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 07:08:12.361938 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":206,"threads_started":1}
I20260812 07:08:12.374866 20700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:12.367998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34262 (local address 127.11.192.253:36785)
0812 07:08:12.368210 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:12.368216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:12.369101 (+ 885us) server_negotiation.cc:407] Connection header received
0812 07:08:12.369376 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:12.369382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:12.369486 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:12.369561 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:12.370300 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:12.371397 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:12.373161 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:12.373553 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:12.373658 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:12.374183 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:12.374306 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:12.374384 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:12.374528 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 07:08:12.374603 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:08:12.375532 20709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:12.369367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50388 (local address 127.11.192.252:42413)
0812 07:08:12.369715 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:12.369721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:12.369819 (+ 98us) server_negotiation.cc:407] Connection header received
0812 07:08:12.370104 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:12.370110 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:12.370187 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:12.370274 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:12.370876 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:12.371625 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:12.373728 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:12.374145 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:12.374260 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:12.374743 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:12.374864 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:12.374927 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:12.375196 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 07:08:12.375275 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234}
I20260812 07:08:12.376513 20726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:12.367968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53916 (local address 127.11.192.254:46595)
0812 07:08:12.368589 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:12.368595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:12.369018 (+ 423us) server_negotiation.cc:407] Connection header received
0812 07:08:12.369295 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:12.369302 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:12.369599 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:12.369782 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:12.370393 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:12.371412 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:12.373666 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:12.374228 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:12.374343 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:12.374924 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:12.375387 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:12.375493 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:12.375633 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 07:08:12.375715 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":366,"threads_started":1}
I20260812 07:08:12.381253 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 20121
I20260812 07:08:12.382136 20235 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:08:12.490267 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 20121
W20260812 07:08:12.520265 20285 connection.cc:535] server connection from 127.0.0.1:34122 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 07:08:12.520246 20405 connection.cc:535] server connection from 127.0.0.1:50340 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 07:08:12.520828 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 20253
I20260812 07:08:12.521876 20355 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:08:12.629608 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 20253
I20260812 07:08:12.655467 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 20377
I20260812 07:08:12.656569 20479 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:08:12.761178 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 20377
2026-08-12T07:08:12Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5165 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:08:12 dist-test-slave-lgr5 krb5kdc[20736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:08:12 dist-test-slave-lgr5 krb5kdc[20736](info): set up 2 sockets
Aug 12 07:08:12 dist-test-slave-lgr5 krb5kdc[20736](info): commencing operation
krb5kdc: starting...
W20260812 07:08:14.059959 12035 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.005s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/test-user.keytab.
Aug 12 07:08:14 dist-test-slave-lgr5 krb5kdc[20736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:08:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:08:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
Entry for principal kudu/127.11.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab.
I20260812 07:08:14.334753 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:39925
--webserver_interface=127.11.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:39747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:39925,127.11.192.253:38101,127.11.192.252:40457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:08:14.653784 20763 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:08:14.654080 20763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:14.654183 20763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:14.654286 20763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:14.661202 20763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:14.661278 20763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:14.661363 20763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:14.661432 20763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:14.661514 20763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:14.670742 20763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:39747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:39925,127.11.192.253:38101,127.11.192.252:40457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.254.keytab
--principal=kudu/127.11.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.254:39925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:08:14.671880 20763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:14.673097 20763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:14.684129 20771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:14.684093 20774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:14.684154 20772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:14.685472 20763 server_base.cc:1034] running on GCE node
I20260812 07:08:14.686158 20763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:14.687403 20763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:14.688616 20763 hybrid_clock.cc:648] HybridClock initialized: now 1786518494688552 us; error 38 us; skew 500 ppm
I20260812 07:08:14.688920 20763 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:08:14 dist-test-slave-lgr5 krb5kdc[20736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518494, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:14.691818 20763 init.cc:377] Logged in from keytab as kudu/127.11.192.254@KRBTEST.COM (short username kudu)
I20260812 07:08:14.693178 20763 webserver.cc:460] Webserver started at http://127.11.192.254:43071/ using document root <none> and password file <none>
I20260812 07:08:14.693802 20763 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:14.693907 20763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:14.694198 20763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:14.697060 20763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-0/data/instance:
uuid: "1e04c81213e64468a9598549eb19196e"
format_stamp: "Formatted at 2026-08-12 07:08:14 on dist-test-slave-lgr5"
server_key: "034b392350f5f004558bf3e0666877c3"
server_key_iv: "64f6121227386ac7069684f758700b85"
server_key_version: "encryptionkey@0"
I20260812 07:08:14.697794 20763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-0/wal/instance:
uuid: "1e04c81213e64468a9598549eb19196e"
format_stamp: "Formatted at 2026-08-12 07:08:14 on dist-test-slave-lgr5"
server_key: "034b392350f5f004558bf3e0666877c3"
server_key_iv: "64f6121227386ac7069684f758700b85"
server_key_version: "encryptionkey@0"
I20260812 07:08:14.702422 20763 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 07:08:14.705581 20785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:14.706717 20763 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:08:14.706871 20763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-0/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-0/wal
uuid: "1e04c81213e64468a9598549eb19196e"
format_stamp: "Formatted at 2026-08-12 07:08:14 on dist-test-slave-lgr5"
server_key: "034b392350f5f004558bf3e0666877c3"
server_key_iv: "64f6121227386ac7069684f758700b85"
server_key_version: "encryptionkey@0"
I20260812 07:08:14.707064 20763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:14.751710 20763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:14.755936 20763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:14.756210 20763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:14.772902 20763 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.254:39925
I20260812 07:08:14.772905 20879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.254:39925 every 8 connection(s)
I20260812 07:08:14.774148 20763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-0/data/info.pb
I20260812 07:08:14.777760 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 20763
I20260812 07:08:14.777796 20883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:14.777966 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-0/wal/instance
I20260812 07:08:14.778548 12035 external_mini_cluster.cc:1428] Setting key 296113097adfda2e7fa1d9ca4c425de9
I20260812 07:08:14.786232 20883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39925 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:08:14 dist-test-slave-lgr5 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518494, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.192.253@KRBTEST.COM; defaulting to no policy
I20260812 07:08:14.812599 20886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:14.789641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51512 (local address 127.11.192.254:39925)
0812 07:08:14.790434 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:14.790447 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:14.790982 (+ 535us) server_negotiation.cc:407] Connection header received
0812 07:08:14.792903 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:14.792910 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:14.793204 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:14.793407 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:14.794866 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:14.796764 (+ 1898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:14.798666 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:14.799967 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:14.800249 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:14.804335 (+ 4086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:14.804346 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:14.804361 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:14.804396 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:14.808070 (+ 3674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:14.809009 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:14.809018 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:14.809021 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:14.809194 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:14.809637 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:14.809641 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:14.809645 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:14.810265 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:14.810488 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:14.811207 (+ 719us) server_negotiation.cc:299] Negotiation successful
0812 07:08:14.811493 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":345,"threads_started":1}
I20260812 07:08:14.812665 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:14.789641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39925 (local address 127.0.0.1:51512)
0812 07:08:14.790814 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:14.790855 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:14.792559 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:14.793448 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:14.793453 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:14.794179 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:14.794634 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:14.794659 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:14.796959 (+ 2300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:14.796965 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:14.798509 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:14.798523 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:14.800278 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:14.800289 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:14.800485 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:14.801196 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:14.801222 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:14.804067 (+ 2845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:14.808224 (+ 4157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:14.808234 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:14.808247 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:14.808853 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:14.809314 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:14.809318 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:14.809322 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:14.809517 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:14.810532 (+ 1015us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:14.810539 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:14.810774 (+ 235us) client_negotiation.cc:769] Sending connection context
0812 07:08:14.811057 (+ 283us) client_negotiation.cc:240] Negotiation successful
0812 07:08:14.811844 (+ 787us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":930,"mutex_wait_us":140,"spinlock_wait_cycles":65792,"thread_start_us":700,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.253@KRBTEST.COM" created.
I20260812 07:08:14.815481 20883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:08:14.816452 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:14.816047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:38101 (local address 127.0.0.1:36802)
0812 07:08:14.816228 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:14.816338 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260812 07:08:14.816715 20786 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.192.253:38101: connect: Connection refused (error 111)
I20260812 07:08:14.818158 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:14.817815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:38101 (local address 127.0.0.1:36816)
0812 07:08:14.817962 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:14.818024 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 07:08:14.818406 20883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.253:38101: Network error: Client connection negotiation failed: client connection to 127.11.192.253:38101: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.253.keytab.
Entry for principal kudu/127.11.192.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.253.keytab.
I20260812 07:08:14.854843 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.253:38101
--webserver_interface=127.11.192.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.253.keytab
--principal=kudu/127.11.192.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:39747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:39925,127.11.192.253:38101,127.11.192.252:40457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:08:14.867708 20883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } attempt: 1
I20260812 07:08:14.868857 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:14.868374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:38101 (local address 127.0.0.1:36830)
0812 07:08:14.868584 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:14.868727 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260812 07:08:14.870141 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:14.869699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:38101 (local address 127.0.0.1:36844)
0812 07:08:14.869867 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:14.869962 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 07:08:14.870407 20883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.253:38101: Network error: Client connection negotiation failed: client connection to 127.11.192.253:38101: connect: Connection refused (error 111)
I20260812 07:08:14.938640 20883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } attempt: 2
I20260812 07:08:14.939916 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:14.939358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:38101 (local address 127.0.0.1:36860)
0812 07:08:14.939643 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:14.939756 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260812 07:08:14.941073 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:14.940659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:38101 (local address 127.0.0.1:36868)
0812 07:08:14.940852 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:14.940932 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 07:08:14.941293 20883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.253:38101: Network error: Client connection negotiation failed: client connection to 127.11.192.253:38101: connect: Connection refused (error 111)
I20260812 07:08:15.032539 20883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } attempt: 3
I20260812 07:08:15.033604 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.033178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:38101 (local address 127.0.0.1:36884)
0812 07:08:15.033375 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.033464 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 07:08:15.034510 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.034167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:38101 (local address 127.0.0.1:36890)
0812 07:08:15.034315 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.034379 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 07:08:15.034677 20883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.253:38101: Network error: Client connection negotiation failed: client connection to 127.11.192.253:38101: connect: Connection refused (error 111)
I20260812 07:08:15.177883 20883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } attempt: 4
I20260812 07:08:15.179011 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.178481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:38101 (local address 127.0.0.1:36902)
0812 07:08:15.178783 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.178869 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260812 07:08:15.179943 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.179608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:38101 (local address 127.0.0.1:36910)
0812 07:08:15.179751 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.179826 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.253:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 07:08:15.180127 20883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.253:38101: Network error: Client connection negotiation failed: client connection to 127.11.192.253:38101: connect: Connection refused (error 111)
W20260812 07:08:15.181936 20893 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:08:15.182242 20893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:15.182337 20893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:15.182448 20893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:15.189513 20893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:15.189599 20893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:15.189690 20893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:15.189755 20893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:15.189846 20893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:15.199352 20893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:39747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:39925,127.11.192.253:38101,127.11.192.252:40457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.253.keytab
--principal=kudu/127.11.192.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.253:38101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.192.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:08:15.200577 20893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:15.201813 20893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:15.212711 20900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:15.212687 20903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:15.212847 20906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:15.214213 20893 server_base.cc:1034] running on GCE node
I20260812 07:08:15.214915 20893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:15.216254 20893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:15.217479 20893 hybrid_clock.cc:648] HybridClock initialized: now 1786518495217398 us; error 45 us; skew 500 ppm
I20260812 07:08:15.217773 20893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:08:15 dist-test-slave-lgr5 krb5kdc[20736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518495, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:15.220688 20893 init.cc:377] Logged in from keytab as kudu/127.11.192.253@KRBTEST.COM (short username kudu)
I20260812 07:08:15.221976 20893 webserver.cc:460] Webserver started at http://127.11.192.253:39347/ using document root <none> and password file <none>
I20260812 07:08:15.222630 20893 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:15.222728 20893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:15.223037 20893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:15.225931 20893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-1/data/instance:
uuid: "7fb5f739a96e487daa18dea07ded48f9"
format_stamp: "Formatted at 2026-08-12 07:08:15 on dist-test-slave-lgr5"
server_key: "924c43290df40d01cf8b1d7465653d2a"
server_key_iv: "5e4fc6ca452c25fbb87c92d948979c6a"
server_key_version: "encryptionkey@0"
I20260812 07:08:15.226737 20893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-1/wal/instance:
uuid: "7fb5f739a96e487daa18dea07ded48f9"
format_stamp: "Formatted at 2026-08-12 07:08:15 on dist-test-slave-lgr5"
server_key: "924c43290df40d01cf8b1d7465653d2a"
server_key_iv: "5e4fc6ca452c25fbb87c92d948979c6a"
server_key_version: "encryptionkey@0"
I20260812 07:08:15.231566 20893 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:08:15.234814 20916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:15.236038 20893 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:08:15.236204 20893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-1/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-1/wal
uuid: "7fb5f739a96e487daa18dea07ded48f9"
format_stamp: "Formatted at 2026-08-12 07:08:15 on dist-test-slave-lgr5"
server_key: "924c43290df40d01cf8b1d7465653d2a"
server_key_iv: "5e4fc6ca452c25fbb87c92d948979c6a"
server_key_version: "encryptionkey@0"
I20260812 07:08:15.236377 20893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:15.264140 20893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:15.267918 20893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:15.268153 20893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:15.285346 20893 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.253:38101
I20260812 07:08:15.285365 21007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.253:38101 every 8 connection(s)
I20260812 07:08:15.286665 20893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-1/data/info.pb
I20260812 07:08:15.290313 21008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:15.296749 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 20893
I20260812 07:08:15.296926 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-1/wal/instance
I20260812 07:08:15.296679 21008 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39925 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:08:15.297343 12035 external_mini_cluster.cc:1428] Setting key b866690327de272be5a1375e4f4f1700
Aug 12 07:08:15 dist-test-slave-lgr5 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518495, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.253@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:08:15.315896 20886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.299287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51514 (local address 127.11.192.254:39925)
0812 07:08:15.299455 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:15.299459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:15.300716 (+ 1257us) server_negotiation.cc:407] Connection header received
0812 07:08:15.302490 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:15.302496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:15.302581 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:15.302671 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:15.304123 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.304790 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.306252 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.306654 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.306764 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:15.309972 (+ 3208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:15.309977 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:15.309981 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:15.310014 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:15.312984 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:15.313759 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:15.313764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:15.313766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:15.313894 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:15.314305 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:15.314308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:15.314310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:15.314567 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:15.314679 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:15.315517 (+ 838us) server_negotiation.cc:299] Negotiation successful
0812 07:08:15.315679 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:08:15.317116 21011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.299814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39925 (local address 127.0.0.1:51514)
0812 07:08:15.300564 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.300600 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:15.302224 (+ 1624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:15.302915 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:15.302922 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:15.303411 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:15.303932 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:15.303949 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.304943 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.304946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:15.306101 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:15.306114 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.306876 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.306883 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:15.307099 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:15.307690 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:15.307717 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:15.309768 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:15.313112 (+ 3344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:15.313120 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:15.313134 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:15.313644 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:15.314003 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:15.314008 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:15.314013 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:15.314219 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:15.314685 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:15.314693 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:15.315117 (+ 424us) client_negotiation.cc:769] Sending connection context
0812 07:08:15.315438 (+ 321us) client_negotiation.cc:240] Negotiation successful
0812 07:08:15.315866 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":318,"threads_started":1}
I20260812 07:08:15.318670 21008 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:08:15 dist-test-slave-lgr5 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518495, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.253@KRBTEST.COM for kudu/127.11.192.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.192.252@KRBTEST.COM; defaulting to no policy
I20260812 07:08:15.333052 21011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.319370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:38101 (local address 127.0.0.1:36916)
0812 07:08:15.319539 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.319557 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:15.319647 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:15.320430 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:15.320439 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:15.320871 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:15.321037 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:15.321047 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.322336 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.322340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:15.323483 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:15.323499 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.324276 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.324288 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:15.324439 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:15.324930 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:15.324949 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:15.327177 (+ 2228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:15.330887 (+ 3710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:15.330891 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:15.330893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:15.331228 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:15.331640 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:15.331644 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:15.331646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:15.331772 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:15.332404 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:15.332407 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:15.332530 (+ 123us) client_negotiation.cc:769] Sending connection context
0812 07:08:15.332634 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 07:08:15.332809 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 07:08:15.333304 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.319484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36916 (local address 127.11.192.253:38101)
0812 07:08:15.320008 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:15.320014 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:15.320038 (+ 24us) server_negotiation.cc:407] Connection header received
0812 07:08:15.320115 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:15.320123 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:15.320281 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:15.320389 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:15.321191 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.322196 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.323607 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.324132 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.324232 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:15.327385 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:15.327399 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:15.327414 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:15.327455 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:15.330757 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:15.331361 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:15.331371 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:15.331374 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:15.331518 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:15.331861 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:15.331865 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:15.331867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:15.332286 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:15.332440 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:15.332751 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 07:08:15.332922 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":255,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.192.252@KRBTEST.COM" created.
I20260812 07:08:15.335435 21008 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:08:15.336447 21011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.335952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:40457 (local address 127.0.0.1:56572)
0812 07:08:15.336135 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.336282 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 07:08:15.336710 20918 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
I20260812 07:08:15.338130 21011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.337771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:40457 (local address 127.0.0.1:56588)
0812 07:08:15.337914 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.338016 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 07:08:15.338326 21008 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:40457: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.192.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.252.keytab.
Entry for principal kudu/127.11.192.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.252.keytab.
I20260812 07:08:15.365525 12035 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
/tmp/dist-test-taskpv3LkS/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.252:40457
--webserver_interface=127.11.192.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.252.keytab
--principal=kudu/127.11.192.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.192.212:39747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.192.254:39925,127.11.192.253:38101,127.11.192.252:40457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:08:15.387647 21008 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } attempt: 1
I20260812 07:08:15.388844 21011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.388369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:40457 (local address 127.0.0.1:56600)
0812 07:08:15.388569 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.388682 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 07:08:15.389856 21011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.389482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:40457 (local address 127.0.0.1:56612)
0812 07:08:15.389653 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.389738 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 07:08:15.390084 21008 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:40457: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
I20260812 07:08:15.458280 21008 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } attempt: 2
I20260812 07:08:15.459293 21011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.458862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:40457 (local address 127.0.0.1:56618)
0812 07:08:15.459050 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.459139 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 07:08:15.460254 21011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.459887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:40457 (local address 127.0.0.1:56634)
0812 07:08:15.460032 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.460140 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 07:08:15.460444 21008 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:40457: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
I20260812 07:08:15.479351 20883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } attempt: 5
Aug 12 07:08:15 dist-test-slave-lgr5 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518494, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for kudu/127.11.192.253@KRBTEST.COM
I20260812 07:08:15.493076 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.479999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:38101 (local address 127.0.0.1:36922)
0812 07:08:15.480303 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.480322 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:15.480449 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:15.480805 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:15.480810 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:15.481304 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:15.481548 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:15.481562 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.482519 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.482524 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:15.483863 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:15.483879 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.484789 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.484797 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:15.485037 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:15.485648 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:15.485672 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:15.487996 (+ 2324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:15.490784 (+ 2788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:15.490790 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:15.490794 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:15.491155 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:15.491500 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:15.491505 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:15.491509 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:15.491640 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:15.492163 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:15.492168 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:15.492309 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 07:08:15.492434 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 07:08:15.492716 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182}
I20260812 07:08:15.493078 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.480222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36922 (local address 127.11.192.253:38101)
0812 07:08:15.480391 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:15.480397 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:15.480416 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:08:15.480567 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:15.480572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:15.480652 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:15.480739 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:15.481694 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.482387 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.483999 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.484609 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.484720 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:15.488192 (+ 3472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:15.488197 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:15.488200 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:15.488231 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:15.490658 (+ 2427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:15.491269 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:15.491274 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:15.491276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:15.491395 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:15.491746 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:15.491752 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:15.491755 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:15.492025 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:15.492186 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:15.492562 (+ 376us) server_negotiation.cc:299] Negotiation successful
0812 07:08:15.492764 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 07:08:15.494362 20883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:08:15.495240 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.494894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:40457 (local address 127.0.0.1:56648)
0812 07:08:15.495038 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.495118 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 07:08:15.496137 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.495776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:40457 (local address 127.0.0.1:56656)
0812 07:08:15.495920 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.495985 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 07:08:15.496346 20883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:40457: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
I20260812 07:08:15.547544 20883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } attempt: 1
I20260812 07:08:15.548619 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.548073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:40457 (local address 127.0.0.1:56670)
0812 07:08:15.548309 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.548421 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 07:08:15.549589 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.549216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:40457 (local address 127.0.0.1:56684)
0812 07:08:15.549356 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.549443 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 07:08:15.549796 20883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:40457: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
I20260812 07:08:15.551637 21008 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } attempt: 3
I20260812 07:08:15.552497 21011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.552100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:40457 (local address 127.0.0.1:56692)
0812 07:08:15.552284 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.552361 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 07:08:15.553449 21011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.553065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:40457 (local address 127.0.0.1:56696)
0812 07:08:15.553275 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.553346 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260812 07:08:15.553684 21008 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:40457: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
I20260812 07:08:15.617990 20883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } attempt: 2
I20260812 07:08:15.619089 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.618630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:40457 (local address 127.0.0.1:56706)
0812 07:08:15.618831 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.618918 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 07:08:15.619961 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.619685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:40457 (local address 127.0.0.1:56720)
0812 07:08:15.619798 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.619873 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 07:08:15.620195 20883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:40457: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
W20260812 07:08:15.687868 21020 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:08:15.688189 21020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:08:15.688279 21020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:08:15.688411 21020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:08:15.695516 21020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:08:15.695591 21020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:08:15.695676 21020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:08:15.695735 21020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:08:15.695816 21020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:08:15.696862 21008 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } attempt: 4
I20260812 07:08:15.697934 21011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.697504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:40457 (local address 127.0.0.1:56728)
0812 07:08:15.697699 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.697783 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 07:08:15.698915 21011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.698619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:40457 (local address 127.0.0.1:56732)
0812 07:08:15.698745 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.698814 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 07:08:15.699095 21008 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:40457: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
I20260812 07:08:15.705674 21020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.192.212:39747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.192.254:39925,127.11.192.253:38101,127.11.192.252:40457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/krb5kdc/kudu_127.11.192.252.keytab
--principal=kudu/127.11.192.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.192.252:40457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.192.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:08:15.706785 21020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:08:15.707976 21020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:08:15.718992 21034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:15.719079 21027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:08:15.719084 21028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:08:15.720010 21020 server_base.cc:1034] running on GCE node
I20260812 07:08:15.720714 21020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:08:15.721930 21020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:08:15.723143 21020 hybrid_clock.cc:648] HybridClock initialized: now 1786518495723083 us; error 34 us; skew 500 ppm
I20260812 07:08:15.723433 21020 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:08:15.726373 20883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } attempt: 3
I20260812 07:08:15.727237 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.726854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:40457 (local address 127.0.0.1:56740)
0812 07:08:15.727016 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.727093 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 07:08:15.728048 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.727750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:40457 (local address 127.0.0.1:56748)
0812 07:08:15.727875 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.727947 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 07:08:15.728241 20883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.192.252:40457: Network error: Client connection negotiation failed: client connection to 127.11.192.252:40457: connect: Connection refused (error 111)
Aug 12 07:08:15 dist-test-slave-lgr5 krb5kdc[20736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518495, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:08:15.738724 21020 init.cc:377] Logged in from keytab as kudu/127.11.192.252@KRBTEST.COM (short username kudu)
I20260812 07:08:15.740109 21020 webserver.cc:460] Webserver started at http://127.11.192.252:43223/ using document root <none> and password file <none>
I20260812 07:08:15.740796 21020 fs_manager.cc:359] Metadata directory not provided
I20260812 07:08:15.740904 21020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:08:15.741178 21020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:08:15.743911 21020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-2/data/instance:
uuid: "4579b9b85afb4c178fd4ebab8ac4bccd"
format_stamp: "Formatted at 2026-08-12 07:08:15 on dist-test-slave-lgr5"
server_key: "b20704c369ddf6e543a94564c02123fb"
server_key_iv: "b4b7e0c62597e8816f20c61f9d4828c2"
server_key_version: "encryptionkey@0"
I20260812 07:08:15.744700 21020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-2/wal/instance:
uuid: "4579b9b85afb4c178fd4ebab8ac4bccd"
format_stamp: "Formatted at 2026-08-12 07:08:15 on dist-test-slave-lgr5"
server_key: "b20704c369ddf6e543a94564c02123fb"
server_key_iv: "b4b7e0c62597e8816f20c61f9d4828c2"
server_key_version: "encryptionkey@0"
I20260812 07:08:15.749542 21020 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260812 07:08:15.752665 21047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:08:15.753842 21020 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:08:15.754007 21020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-2/data,/tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-2/wal
uuid: "4579b9b85afb4c178fd4ebab8ac4bccd"
format_stamp: "Formatted at 2026-08-12 07:08:15 on dist-test-slave-lgr5"
server_key: "b20704c369ddf6e543a94564c02123fb"
server_key_iv: "b4b7e0c62597e8816f20c61f9d4828c2"
server_key_version: "encryptionkey@0"
I20260812 07:08:15.754196 21020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:08:15.771586 21020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:08:15.775377 21020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:08:15.775617 21020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:08:15.792531 21020 rpc_server.cc:307] RPC server started. Bound to: 127.11.192.252:40457
I20260812 07:08:15.792551 21149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.192.252:40457 every 8 connection(s)
I20260812 07:08:15.793758 21020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-2/data/info.pb
I20260812 07:08:15.797211 21150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:08:15.800872 12035 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu as pid 21020
I20260812 07:08:15.801047 12035 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpv3LkS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518136653111-12035-0/minicluster-data/master-2/wal/instance
I20260812 07:08:15.801470 12035 external_mini_cluster.cc:1428] Setting key 982d2ee943f7dccf69836f4eea0b09d1
I20260812 07:08:15.803851 21150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39925 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:08:15 dist-test-slave-lgr5 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.252@KRBTEST.COM
Aug 12 07:08:15 dist-test-slave-lgr5 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518495, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.252@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:08:15.822026 20886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.807559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51522 (local address 127.11.192.254:39925)
0812 07:08:15.807755 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:15.807762 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:15.808297 (+ 535us) server_negotiation.cc:407] Connection header received
0812 07:08:15.808567 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:15.808573 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:15.808651 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:15.808752 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:15.809902 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.810648 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.812231 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.812755 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.812861 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:15.816763 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:15.816768 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:15.816771 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:15.816803 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:15.819111 (+ 2308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:15.819749 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:15.819755 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:15.819758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:15.819907 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:15.820339 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:15.820342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:15.820344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:15.820602 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:15.820726 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:15.821385 (+ 659us) server_negotiation.cc:299] Negotiation successful
0812 07:08:15.821755 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:08:15.822858 21157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.807427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.254:39925 (local address 127.0.0.1:51522)
0812 07:08:15.808154 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.808200 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:15.808420 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:15.808817 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:15.808821 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:15.809459 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:15.809767 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:15.809781 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.810797 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.810803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:15.812105 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:15.812121 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.812978 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.812986 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:15.813163 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:15.813727 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:15.813757 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:15.816586 (+ 2829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:15.819228 (+ 2642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:15.819239 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:15.819249 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:15.819621 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:15.820024 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:15.820029 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:15.820033 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:15.820225 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:15.820747 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:15.820754 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:15.820962 (+ 208us) client_negotiation.cc:769] Sending connection context
0812 07:08:15.821250 (+ 288us) client_negotiation.cc:240] Negotiation successful
0812 07:08:15.821749 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":121,"thread_start_us":341,"threads_started":1}
I20260812 07:08:15.822878 21156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.803041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56758 (local address 127.11.192.252:40457)
0812 07:08:15.803726 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:15.803738 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:15.803776 (+ 38us) server_negotiation.cc:407] Connection header received
0812 07:08:15.805756 (+ 1980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:15.805779 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:15.806151 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:15.806459 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:15.808006 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.809653 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.811129 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.811775 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.811903 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:15.816451 (+ 4548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:15.816462 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:15.816475 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:15.816525 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:15.819146 (+ 2621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:15.819782 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:15.819788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:15.819790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:15.819924 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:15.820319 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:15.820323 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:15.820325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:15.820814 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:15.821100 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:15.821315 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 07:08:15.821609 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":503,"thread_start_us":312,"threads_started":1}
I20260812 07:08:15.824364 21150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:08:15 dist-test-slave-lgr5 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518495, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.252@KRBTEST.COM for kudu/127.11.192.253@KRBTEST.COM
Aug 12 07:08:15 dist-test-slave-lgr5 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.254@KRBTEST.COM
I20260812 07:08:15.836613 21157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.824967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.253:38101 (local address 127.0.0.1:36924)
0812 07:08:15.825117 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.825136 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:15.825225 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:15.825561 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:15.825566 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:15.825950 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:15.826135 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:15.826147 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.827052 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.827056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:15.827908 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:15.827920 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.828649 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.828654 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:15.828850 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:15.829294 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:15.829310 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:15.831346 (+ 2036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:15.834450 (+ 3104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:15.834455 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:15.834459 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:15.834787 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:15.835146 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:15.835151 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:15.835154 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:15.835280 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:15.835754 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:15.835759 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:15.835899 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 07:08:15.836022 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 07:08:15.836288 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 07:08:15.836723 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.825063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36924 (local address 127.11.192.253:38101)
0812 07:08:15.825261 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:15.825266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:15.825285 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:08:15.825346 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:15.825351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:15.825433 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:15.825509 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:15.826267 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.826913 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.828053 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.828517 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.828626 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:15.831535 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:15.831540 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:15.831543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:15.831575 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:15.834330 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:15.834898 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:15.834902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:15.834904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:15.835031 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:15.835381 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:15.835384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:15.835386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:15.835632 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:15.835768 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:15.836154 (+ 386us) server_negotiation.cc:299] Negotiation successful
0812 07:08:15.836367 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:08:15.837528 21150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:08:15.838989 20886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.825850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51536 (local address 127.11.192.254:39925)
0812 07:08:15.825989 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:15.825992 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:15.826006 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:08:15.826167 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:15.826170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:15.826219 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:15.826298 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:15.827470 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.828189 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.829325 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.829720 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.829825 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:15.833935 (+ 4110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:15.833939 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:15.833942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:15.833971 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:15.836328 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:15.837011 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:15.837016 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:15.837019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:15.837176 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:15.837596 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:15.837601 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:15.837604 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:15.837924 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:15.838179 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:15.838490 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 07:08:15.838695 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 07:08:15 dist-test-slave-lgr5 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518495, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.252@KRBTEST.COM for kudu/127.11.192.252@KRBTEST.COM
Aug 12 07:08:15 dist-test-slave-lgr5 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.192.253@KRBTEST.COM
I20260812 07:08:15.850907 21157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.838126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:40457 (local address 127.0.0.1:56772)
0812 07:08:15.838256 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.838269 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:15.838344 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:15.838876 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:15.838881 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:15.839286 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:15.839415 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:15.839425 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.840493 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.840498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:15.841745 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:15.841761 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.842488 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.842492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:15.842648 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:15.843119 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:15.843134 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:15.845382 (+ 2248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:15.848668 (+ 3286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:15.848673 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:15.848676 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:15.849151 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:15.849498 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:15.849502 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:15.849506 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:15.849674 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:15.850153 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:15.850158 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:15.850299 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 07:08:15.850426 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 07:08:15.850613 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 07:08:15.851192 21156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.838472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56772 (local address 127.11.192.252:40457)
0812 07:08:15.838625 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:15.838630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:15.838645 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:08:15.838687 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:15.838691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:15.838750 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:15.838837 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:15.839573 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.840356 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.841880 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.842346 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.842466 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:15.845592 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:15.845598 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:15.845602 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:15.845636 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:15.848536 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:15.849262 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:15.849267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:15.849271 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:15.849387 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:15.849781 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:15.849784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:15.849786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:15.850037 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:15.850142 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:15.850541 (+ 399us) server_negotiation.cc:299] Negotiation successful
0812 07:08:15.850762 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:08:15.855387 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.840469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36938 (local address 127.11.192.253:38101)
0812 07:08:15.840644 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:15.840650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:15.840667 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:08:15.840730 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:15.840733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:15.840780 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:15.840848 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:15.842186 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.843093 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.844706 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.845321 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.845452 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:15.850225 (+ 4773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:15.850229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:15.850232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:15.850260 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:15.853065 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:15.853766 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:15.853769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:15.853771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:15.853890 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:15.854253 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:15.854256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:15.854257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:15.854473 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:15.854656 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:15.854922 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 07:08:15.855099 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 07:08:15.856498 21150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4579b9b85afb4c178fd4ebab8ac4bccd: Bootstrap starting.
I20260812 07:08:15.859840 12035 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:08:15.859959 21150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4579b9b85afb4c178fd4ebab8ac4bccd: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:15.861212 21150 log.cc:824] T 00000000000000000000000000000000 P 4579b9b85afb4c178fd4ebab8ac4bccd: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:15.862707 20833 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "1e04c81213e64468a9598549eb19196e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51536
I20260812 07:08:15.865545 21150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4579b9b85afb4c178fd4ebab8ac4bccd: No bootstrap required, opened a new log
I20260812 07:08:15.872340 21150 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4579b9b85afb4c178fd4ebab8ac4bccd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e04c81213e64468a9598549eb19196e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39925 } } peers { permanent_uuid: "7fb5f739a96e487daa18dea07ded48f9" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } } peers { permanent_uuid: "4579b9b85afb4c178fd4ebab8ac4bccd" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } }
I20260812 07:08:15.872620 21150 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4579b9b85afb4c178fd4ebab8ac4bccd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4579b9b85afb4c178fd4ebab8ac4bccd, State: Initialized, Role: FOLLOWER
I20260812 07:08:15.873039 21150 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4579b9b85afb4c178fd4ebab8ac4bccd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e04c81213e64468a9598549eb19196e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39925 } } peers { permanent_uuid: "7fb5f739a96e487daa18dea07ded48f9" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } } peers { permanent_uuid: "4579b9b85afb4c178fd4ebab8ac4bccd" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } }
I20260812 07:08:15.873894 21160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4579b9b85afb4c178fd4ebab8ac4bccd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e04c81213e64468a9598549eb19196e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39925 } } peers { permanent_uuid: "7fb5f739a96e487daa18dea07ded48f9" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } } peers { permanent_uuid: "4579b9b85afb4c178fd4ebab8ac4bccd" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } } }
I20260812 07:08:15.874265 21160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4579b9b85afb4c178fd4ebab8ac4bccd [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:08:15.874943 21150 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4579b9b85afb4c178fd4ebab8ac4bccd [sys.catalog]: configured and running, proceeding with master startup.
W20260812 07:08:15.883183 21177 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4579b9b85afb4c178fd4ebab8ac4bccd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:08:15.883294 21177 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:08:15.881685 21020 master_runner.cc:186] Error getting master registration for 127.11.192.254:39925: OK, instance_id { permanent_uuid: "1e04c81213e64468a9598549eb19196e" instance_seqno: 1786518494768193 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 07:08:15.892309 21020 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 07:08:15.905432 20883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } attempt: 4
Aug 12 07:08:15 dist-test-slave-lgr5 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518494, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.254@KRBTEST.COM for kudu/127.11.192.252@KRBTEST.COM
I20260812 07:08:15.920744 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.906102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:40457 (local address 127.0.0.1:56784)
0812 07:08:15.906298 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.906317 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:15.906439 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:15.906780 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:15.906784 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:15.907201 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:15.907378 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:15.907387 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.908468 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.908474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:15.909480 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:15.909498 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.910215 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.910221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:15.910452 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:15.911094 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:15.911115 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:15.913890 (+ 2775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:15.917870 (+ 3980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:15.917876 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:15.917880 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:15.918402 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:15.918786 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:15.918791 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:15.918795 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:15.918968 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:15.919698 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:15.919704 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:15.919881 (+ 177us) client_negotiation.cc:769] Sending connection context
0812 07:08:15.920050 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 07:08:15.920350 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 07:08:15.920768 21156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.906164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56784 (local address 127.11.192.252:40457)
0812 07:08:15.906341 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:15.906347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:15.906370 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:08:15.906575 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:15.906581 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:15.906648 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:15.906728 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:15.907536 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.908326 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.909621 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.910061 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.910171 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:15.914124 (+ 3953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:15.914130 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:15.914135 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:15.914173 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:15.917727 (+ 3554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:15.918527 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:15.918532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:15.918536 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:15.918675 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:15.919096 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:15.919101 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:15.919104 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:15.919561 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:15.919716 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:15.920189 (+ 473us) server_negotiation.cc:299] Negotiation successful
0812 07:08:15.920408 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:08:15.926529 20883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e: Bootstrap starting.
I20260812 07:08:15.929675 20883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:15.930685 20883 log.cc:824] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:15.933585 20883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e: No bootstrap required, opened a new log
I20260812 07:08:15.938823 20883 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e04c81213e64468a9598549eb19196e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39925 } } peers { permanent_uuid: "7fb5f739a96e487daa18dea07ded48f9" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } } peers { permanent_uuid: "4579b9b85afb4c178fd4ebab8ac4bccd" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } }
I20260812 07:08:15.939210 20883 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e04c81213e64468a9598549eb19196e, State: Initialized, Role: FOLLOWER
I20260812 07:08:15.939648 20883 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e04c81213e64468a9598549eb19196e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39925 } } peers { permanent_uuid: "7fb5f739a96e487daa18dea07ded48f9" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } } peers { permanent_uuid: "4579b9b85afb4c178fd4ebab8ac4bccd" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } }
I20260812 07:08:15.940675 21183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e04c81213e64468a9598549eb19196e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39925 } } peers { permanent_uuid: "7fb5f739a96e487daa18dea07ded48f9" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } } peers { permanent_uuid: "4579b9b85afb4c178fd4ebab8ac4bccd" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } } }
I20260812 07:08:15.941181 21183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:08:15.941830 20883 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:15.941969 20833 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 07:08:15.942217 20833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:15.945822 20833 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e04c81213e64468a9598549eb19196e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39925 } } peers { permanent_uuid: "7fb5f739a96e487daa18dea07ded48f9" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } } peers { permanent_uuid: "4579b9b85afb4c178fd4ebab8ac4bccd" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } }
I20260812 07:08:15.949928 20833 leader_election.cc:288] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e [CANDIDATE]: Term 1 election: Requested vote from peers 7fb5f739a96e487daa18dea07ded48f9 (127.11.192.253:38101), 4579b9b85afb4c178fd4ebab8ac4bccd (127.11.192.252:40457)
I20260812 07:08:15.950950 21102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e04c81213e64468a9598549eb19196e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4579b9b85afb4c178fd4ebab8ac4bccd"
I20260812 07:08:15.951555 21102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4579b9b85afb4c178fd4ebab8ac4bccd [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:15.955577 21102 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4579b9b85afb4c178fd4ebab8ac4bccd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e04c81213e64468a9598549eb19196e in term 1.
I20260812 07:08:15.957894 20786 leader_election.cc:302] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e04c81213e64468a9598549eb19196e, 4579b9b85afb4c178fd4ebab8ac4bccd; no voters:
I20260812 07:08:15.958490 21183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:08:15.960280 21183 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e [term 1 LEADER]: Becoming Leader. State: Replica: 1e04c81213e64468a9598549eb19196e, State: Running, Role: LEADER
I20260812 07:08:15.961015 21183 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e04c81213e64468a9598549eb19196e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39925 } } peers { permanent_uuid: "7fb5f739a96e487daa18dea07ded48f9" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } } peers { permanent_uuid: "4579b9b85afb4c178fd4ebab8ac4bccd" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } }
I20260812 07:08:15.968791 21183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e04c81213e64468a9598549eb19196e. Latest consensus state: current_term: 1 leader_uuid: "1e04c81213e64468a9598549eb19196e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e04c81213e64468a9598549eb19196e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39925 } } peers { permanent_uuid: "7fb5f739a96e487daa18dea07ded48f9" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } } peers { permanent_uuid: "4579b9b85afb4c178fd4ebab8ac4bccd" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } } }
I20260812 07:08:15.969518 21183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:15.958968 20763 master_runner.cc:186] Error getting master registration for 127.11.192.253:38101: OK, instance_id { permanent_uuid: "7fb5f739a96e487daa18dea07ded48f9" instance_seqno: 1786518495280503 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 07:08:15.970513 20763 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 07:08:15.986649 21201 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:08:15.986789 21201 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:08:15.987329 21209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:08:15.989892 21209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:08:15.998309 21008 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } attempt: 5
W20260812 07:08:16.008832 20800 tablet.cc:2369] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:08:16.008951 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.981460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36942 (local address 127.11.192.253:38101)
0812 07:08:15.981678 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:15.981685 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:15.991534 (+ 9849us) server_negotiation.cc:407] Connection header received
0812 07:08:15.991624 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:15.991631 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:15.991725 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:15.991846 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:15.994447 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.995603 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.997538 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:15.998199 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:15.998308 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:16.001678 (+ 3370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:16.001685 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:16.001691 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:16.001734 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:16.005316 (+ 3582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:16.006168 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:16.006175 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:16.006179 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:16.006366 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:16.006869 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:16.006875 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:16.006878 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:16.007279 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:16.007549 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:16.008267 (+ 718us) server_negotiation.cc:299] Negotiation successful
0812 07:08:16.008522 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
Aug 12 07:08:16 dist-test-slave-lgr5 krb5kdc[20736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518495, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.192.253@KRBTEST.COM for kudu/127.11.192.252@KRBTEST.COM
I20260812 07:08:16.026496 21102 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4579b9b85afb4c178fd4ebab8ac4bccd [term 1 FOLLOWER]: Refusing update from remote peer 1e04c81213e64468a9598549eb19196e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260812 07:08:16.027583 21064 tablet.cc:2369] T 00000000000000000000000000000000 P 4579b9b85afb4c178fd4ebab8ac4bccd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:08:16.028342 21183 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4579b9b85afb4c178fd4ebab8ac4bccd" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:16.034104 20886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.981819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51550 (local address 127.11.192.254:39925)
0812 07:08:15.981973 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:15.981979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:15.988621 (+ 6642us) server_negotiation.cc:407] Connection header received
0812 07:08:15.990452 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:15.990458 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:15.990541 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:15.990611 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:16.005730 (+ 15119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.006897 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.011558 (+ 4661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.012592 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.012736 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:16.021489 (+ 8753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:16.021498 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:16.021503 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:16.021555 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:16.025446 (+ 3891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:16.026273 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:16.026280 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:16.026283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:16.026471 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:16.027143 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:16.027149 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:16.027152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:16.027894 (+ 742us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:16.028202 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:16.033482 (+ 5280us) server_negotiation.cc:299] Negotiation successful
0812 07:08:16.033733 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:08:16.038914 21156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.983553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56800 (local address 127.11.192.252:40457)
0812 07:08:15.983718 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:15.983723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:15.993454 (+ 9731us) server_negotiation.cc:407] Connection header received
0812 07:08:16.005377 (+ 11923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.005384 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.005468 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.005564 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:16.011615 (+ 6051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.012624 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.016699 (+ 4075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.017298 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.017388 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:16.028790 (+ 11402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:16.028797 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:16.028802 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:16.028839 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:16.032446 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:16.036477 (+ 4031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:16.036483 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:16.036486 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:16.036642 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:16.037082 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:16.037087 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:16.037090 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:16.037495 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:16.037735 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:16.038286 (+ 551us) server_negotiation.cc:299] Negotiation successful
0812 07:08:16.038528 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54,"spinlock_wait_cycles":10496}
I20260812 07:08:16.041366 21011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.998845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.192.252:40457 (local address 127.0.0.1:56810)
0812 07:08:15.999235 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0812 07:08:15.999252 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:08:15.999345 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:08:16.008861 (+ 9516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:08:16.008866 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:08:16.009328 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:08:16.009534 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:16.009547 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.011248 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.011253 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:16.012601 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:08:16.012617 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.013544 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.013551 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:08:16.013820 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:16.014498 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:08:16.014525 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:08:16.017613 (+ 3088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:08:16.024872 (+ 7259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:16.024879 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:16.024883 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:16.025396 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:16.036318 (+ 10922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:08:16.036324 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:08:16.036328 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:08:16.036560 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:08:16.040148 (+ 3588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:08:16.040154 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:08:16.040358 (+ 204us) client_negotiation.cc:769] Sending connection context
0812 07:08:16.040619 (+ 261us) client_negotiation.cc:240] Negotiation successful
0812 07:08:16.040935 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282}
I20260812 07:08:16.047924 21221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:15.999533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56810 (local address 127.11.192.252:40457)
0812 07:08:16.008539 (+ 9006us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.008545 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.008566 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:08:16.008619 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.008624 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.008692 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.009694 (+ 1002us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:16.009968 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.011101 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.012780 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.013384 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.013523 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:16.018025 (+ 4502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:08:16.018062 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:08:16.018070 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:08:16.018140 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:08:16.024649 (+ 6509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:16.035613 (+ 10964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:16.035619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:16.035659 (+ 40us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:16.036023 (+ 364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:08:16.037122 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:08:16.037195 (+ 73us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:08:16.037247 (+ 52us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:08:16.039659 (+ 2412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:08:16.042049 (+ 2390us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:16.042760 (+ 711us) server_negotiation.cc:299] Negotiation successful
0812 07:08:16.043984 (+ 1224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8856,"thread_start_us":11530,"threads_started":1}
I20260812 07:08:16.050202 21160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4579b9b85afb4c178fd4ebab8ac4bccd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e04c81213e64468a9598549eb19196e. Latest consensus state: current_term: 1 leader_uuid: "1e04c81213e64468a9598549eb19196e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e04c81213e64468a9598549eb19196e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39925 } } peers { permanent_uuid: "7fb5f739a96e487daa18dea07ded48f9" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } } peers { permanent_uuid: "4579b9b85afb4c178fd4ebab8ac4bccd" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } } }
I20260812 07:08:16.050544 21160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4579b9b85afb4c178fd4ebab8ac4bccd [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:08:16.053514 21008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fb5f739a96e487daa18dea07ded48f9: Bootstrap starting.
I20260812 07:08:16.053409 21183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e04c81213e64468a9598549eb19196e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e04c81213e64468a9598549eb19196e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39925 } } peers { permanent_uuid: "7fb5f739a96e487daa18dea07ded48f9" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } } peers { permanent_uuid: "4579b9b85afb4c178fd4ebab8ac4bccd" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } } }
I20260812 07:08:16.053647 21183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:16.056195 21222 mvcc.cc:205] Tried to move back new op lower bound from 7317579759692263424 to 7317579759462506496. Current Snapshot: MvccSnapshot[applied={T|T < 7317579759692263424}]
I20260812 07:08:16.056752 21160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4579b9b85afb4c178fd4ebab8ac4bccd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e04c81213e64468a9598549eb19196e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e04c81213e64468a9598549eb19196e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39925 } } peers { permanent_uuid: "7fb5f739a96e487daa18dea07ded48f9" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } } peers { permanent_uuid: "4579b9b85afb4c178fd4ebab8ac4bccd" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } } }
I20260812 07:08:16.056982 21160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4579b9b85afb4c178fd4ebab8ac4bccd [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:08:16.057921 21008 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7fb5f739a96e487daa18dea07ded48f9: Neither blocks nor log segments found. Creating new log.
I20260812 07:08:16.059203 21008 log.cc:824] T 00000000000000000000000000000000 P 7fb5f739a96e487daa18dea07ded48f9: Log is configured to *not* fsync() on all Append() calls
I20260812 07:08:16.060480 21209 catalog_manager.cc:1347] Generated new cluster ID: 8b3a2eab37044a2a878bf42ca3703235
I20260812 07:08:16.060581 21209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:08:16.066687 21008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fb5f739a96e487daa18dea07ded48f9: No bootstrap required, opened a new log
I20260812 07:08:16.077432 21008 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7fb5f739a96e487daa18dea07ded48f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e04c81213e64468a9598549eb19196e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39925 } } peers { permanent_uuid: "7fb5f739a96e487daa18dea07ded48f9" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } } peers { permanent_uuid: "4579b9b85afb4c178fd4ebab8ac4bccd" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } }
I20260812 07:08:16.077797 21008 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7fb5f739a96e487daa18dea07ded48f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb5f739a96e487daa18dea07ded48f9, State: Initialized, Role: FOLLOWER
I20260812 07:08:16.078284 21008 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fb5f739a96e487daa18dea07ded48f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e04c81213e64468a9598549eb19196e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39925 } } peers { permanent_uuid: "7fb5f739a96e487daa18dea07ded48f9" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } } peers { permanent_uuid: "4579b9b85afb4c178fd4ebab8ac4bccd" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } }
I20260812 07:08:16.099243 21236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fb5f739a96e487daa18dea07ded48f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e04c81213e64468a9598549eb19196e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39925 } } peers { permanent_uuid: "7fb5f739a96e487daa18dea07ded48f9" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } } peers { permanent_uuid: "4579b9b85afb4c178fd4ebab8ac4bccd" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } } }
I20260812 07:08:16.100168 21236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fb5f739a96e487daa18dea07ded48f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:08:16.100859 21008 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7fb5f739a96e487daa18dea07ded48f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:08:16.101133 20962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e04c81213e64468a9598549eb19196e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7fb5f739a96e487daa18dea07ded48f9"
I20260812 07:08:16.101656 20962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7fb5f739a96e487daa18dea07ded48f9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:08:16.109149 20962 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7fb5f739a96e487daa18dea07ded48f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e04c81213e64468a9598549eb19196e in term 1.
I20260812 07:08:16.110008 20961 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7fb5f739a96e487daa18dea07ded48f9 [term 1 FOLLOWER]: Refusing update from remote peer 1e04c81213e64468a9598549eb19196e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:08:16.112150 21183 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fb5f739a96e487daa18dea07ded48f9" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:08:16.118546 21209 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:08:16.123468 21209 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:08:16.165565 21236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fb5f739a96e487daa18dea07ded48f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e04c81213e64468a9598549eb19196e. Latest consensus state: current_term: 1 leader_uuid: "1e04c81213e64468a9598549eb19196e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e04c81213e64468a9598549eb19196e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39925 } } peers { permanent_uuid: "7fb5f739a96e487daa18dea07ded48f9" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } } peers { permanent_uuid: "4579b9b85afb4c178fd4ebab8ac4bccd" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } } }
I20260812 07:08:16.165887 21236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fb5f739a96e487daa18dea07ded48f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:08:16.167232 21250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fb5f739a96e487daa18dea07ded48f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e04c81213e64468a9598549eb19196e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e04c81213e64468a9598549eb19196e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39925 } } peers { permanent_uuid: "7fb5f739a96e487daa18dea07ded48f9" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } } peers { permanent_uuid: "4579b9b85afb4c178fd4ebab8ac4bccd" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } } }
I20260812 07:08:16.167701 21250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fb5f739a96e487daa18dea07ded48f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:08:16.168814 21206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e04c81213e64468a9598549eb19196e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e04c81213e64468a9598549eb19196e" member_type: VOTER last_known_addr { host: "127.11.192.254" port: 39925 } } peers { permanent_uuid: "7fb5f739a96e487daa18dea07ded48f9" member_type: VOTER last_known_addr { host: "127.11.192.253" port: 38101 } } peers { permanent_uuid: "4579b9b85afb4c178fd4ebab8ac4bccd" member_type: VOTER last_known_addr { host: "127.11.192.252" port: 40457 } } }
I20260812 07:08:16.169236 21206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e [sys.catalog]: This master's current role is: LEADER
I20260812 07:08:16.172565 21209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e04c81213e64468a9598549eb19196e: Generated new TSK 0
I20260812 07:08:16.174270 21209 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:08:16.202745 21262 catalog_manager.cc:1259] Loaded cluster ID: 8b3a2eab37044a2a878bf42ca3703235
I20260812 07:08:16.202920 21262 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7fb5f739a96e487daa18dea07ded48f9: loading cluster ID for follower catalog manager: success
I20260812 07:08:16.206625 21262 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7fb5f739a96e487daa18dea07ded48f9: acquiring CA information for follower catalog manager: success
I20260812 07:08:16.208998 21262 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7fb5f739a96e487daa18dea07ded48f9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 07:08:16.341832 20886 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:16.327294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51554 (local address 127.11.192.254:39925)
0812 07:08:16.327501 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.327507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.336394 (+ 8887us) server_negotiation.cc:407] Connection header received
0812 07:08:16.337902 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.337908 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.338027 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.338117 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:16.341554 (+ 3437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51554: BlockingRecv error: recv got EOF from 127.0.0.1:51554 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260812 07:08:16.342864 21015 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:16.322624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36946 (local address 127.11.192.253:38101)
0812 07:08:16.322831 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.322838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.335906 (+ 13068us) server_negotiation.cc:407] Connection header received
0812 07:08:16.338348 (+ 2442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.338354 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.338475 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.338551 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:16.342644 (+ 4093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36946: BlockingRecv error: recv got EOF from 127.0.0.1:36946 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260812 07:08:16.349674 21221 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:16.337069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56826 (local address 127.11.192.252:40457)
0812 07:08:16.337262 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.337269 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.343541 (+ 6272us) server_negotiation.cc:407] Connection header received
0812 07:08:16.343733 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.343740 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.343840 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.343938 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:16.349445 (+ 5507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56826: BlockingRecv error: recv got EOF from 127.0.0.1:56826 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:08:16.396240 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:16.378355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36950 (local address 127.11.192.253:38101)
0812 07:08:16.378537 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.378544 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.388644 (+ 10100us) server_negotiation.cc:407] Connection header received
0812 07:08:16.389000 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.389006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.389211 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.389347 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:16.390070 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.391206 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.393221 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.393873 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.393979 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:16.394733 (+ 754us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:16.395404 (+ 671us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:16.395545 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:16.395787 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 07:08:16.395876 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260812 07:08:16.438690 21221 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:16.430267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56836 (local address 127.11.192.252:40457)
0812 07:08:16.430465 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.430472 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.436609 (+ 6137us) server_negotiation.cc:407] Connection header received
0812 07:08:16.437004 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.437012 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.437146 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.437234 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:16.438390 (+ 1156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56836: BlockingRecv error: recv got EOF from 127.0.0.1:56836 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:08:16.455632 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:16.448447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36964 (local address 127.11.192.253:38101)
0812 07:08:16.448641 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.448647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.449160 (+ 513us) server_negotiation.cc:407] Connection header received
0812 07:08:16.449377 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.449384 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.449517 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.449621 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:16.450141 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.451335 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.453322 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.454087 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.454233 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:16.454669 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:16.454847 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:16.454952 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:16.455157 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 07:08:16.455246 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260812 07:08:16.470201 21221 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:16.463386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56840 (local address 127.11.192.252:40457)
0812 07:08:16.463584 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.463591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.468653 (+ 5062us) server_negotiation.cc:407] Connection header received
0812 07:08:16.468856 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.468863 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.468958 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.469071 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:16.469980 (+ 909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56840: BlockingRecv error: recv got EOF from 127.0.0.1:56840 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:08:16.483358 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:16.476269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36978 (local address 127.11.192.253:38101)
0812 07:08:16.476474 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.476479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.477006 (+ 527us) server_negotiation.cc:407] Connection header received
0812 07:08:16.477188 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.477193 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.477282 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.477393 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:16.477947 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.479168 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.481288 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.481933 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.482082 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:16.482555 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:16.482699 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:16.482794 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:16.482944 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 07:08:16.483034 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260812 07:08:16.497141 21221 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:16.490761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56846 (local address 127.11.192.252:40457)
0812 07:08:16.490937 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.490943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.495577 (+ 4634us) server_negotiation.cc:407] Connection header received
0812 07:08:16.495779 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.495786 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.495903 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.496016 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:16.496897 (+ 881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56846: BlockingRecv error: recv got EOF from 127.0.0.1:56846 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:08:16.510707 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:16.503993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36988 (local address 127.11.192.253:38101)
0812 07:08:16.504245 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.504251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.504745 (+ 494us) server_negotiation.cc:407] Connection header received
0812 07:08:16.504908 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.504914 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.505000 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.505087 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:16.505580 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.506664 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.508638 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.509405 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.509552 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:16.509992 (+ 440us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:16.510132 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:16.510302 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:16.510360 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 07:08:16.510435 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
W20260812 07:08:16.520146 21221 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:16.517888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56854 (local address 127.11.192.252:40457)
0812 07:08:16.518143 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.518148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.518662 (+ 514us) server_negotiation.cc:407] Connection header received
0812 07:08:16.518845 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.518853 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.518949 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.519062 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:16.519880 (+ 818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56854: BlockingRecv error: recv got EOF from 127.0.0.1:56854 (error 108)
Metrics: {"server-negotiator.queue_time_us":145}
I20260812 07:08:16.537581 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:16.530906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37004 (local address 127.11.192.253:38101)
0812 07:08:16.531066 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.531071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.531547 (+ 476us) server_negotiation.cc:407] Connection header received
0812 07:08:16.531694 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.531700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.531787 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.531892 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:16.532327 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.533459 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.535314 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.536069 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.536247 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:16.536672 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:16.536848 (+ 176us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:16.536955 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:16.537084 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 07:08:16.537173 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260812 07:08:16.546852 21221 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:16.544459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56864 (local address 127.11.192.252:40457)
0812 07:08:16.544655 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.544662 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.545223 (+ 561us) server_negotiation.cc:407] Connection header received
0812 07:08:16.545490 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.545497 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.545593 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.545694 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:16.546612 (+ 918us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56864: BlockingRecv error: recv got EOF from 127.0.0.1:56864 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 07:08:16.573892 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:16.567173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37012 (local address 127.11.192.253:38101)
0812 07:08:16.567379 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.567386 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.568006 (+ 620us) server_negotiation.cc:407] Connection header received
0812 07:08:16.568262 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.568268 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.568349 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.568440 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:16.568965 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.570156 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.572140 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.572606 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.572701 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:16.573242 (+ 541us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:16.573340 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:16.573407 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:16.573567 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 07:08:16.573631 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260812 07:08:16.582556 21221 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:16.580314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56870 (local address 127.11.192.252:40457)
0812 07:08:16.580526 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.580532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.581070 (+ 538us) server_negotiation.cc:407] Connection header received
0812 07:08:16.581265 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.581270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.581340 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.581415 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:16.582313 (+ 898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56870: BlockingRecv error: recv got EOF from 127.0.0.1:56870 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:08:16.624261 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:16.618326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37028 (local address 127.11.192.253:38101)
0812 07:08:16.618570 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.618577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.618930 (+ 353us) server_negotiation.cc:407] Connection header received
0812 07:08:16.619098 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.619105 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.619226 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.619339 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:16.619691 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.620907 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.622179 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.622914 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.623073 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:16.623389 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:16.623533 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:16.623656 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:16.623773 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 07:08:16.623873 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260812 07:08:16.632911 21221 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:16.630896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56884 (local address 127.11.192.252:40457)
0812 07:08:16.631058 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.631062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.631465 (+ 403us) server_negotiation.cc:407] Connection header received
0812 07:08:16.631652 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.631656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.631723 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.631797 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:16.632681 (+ 884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56884: BlockingRecv error: recv got EOF from 127.0.0.1:56884 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 07:08:16.705262 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:16.700176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37034 (local address 127.11.192.253:38101)
0812 07:08:16.700362 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.700368 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.700750 (+ 382us) server_negotiation.cc:407] Connection header received
0812 07:08:16.700966 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.700971 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.701051 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.701127 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:16.701553 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.702323 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.703648 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.704034 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.704202 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:16.704612 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:16.704713 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:16.704787 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:16.704934 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 07:08:16.705021 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 07:08:16.711915 21221 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:16.710249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56900 (local address 127.11.192.252:40457)
0812 07:08:16.710402 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.710407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.710783 (+ 376us) server_negotiation.cc:407] Connection header received
0812 07:08:16.710936 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.710941 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.711012 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.711100 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:16.711720 (+ 620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56900: BlockingRecv error: recv got EOF from 127.0.0.1:56900 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:08:16.848436 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:16.843447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37040 (local address 127.11.192.253:38101)
0812 07:08:16.843677 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.843683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.843970 (+ 287us) server_negotiation.cc:407] Connection header received
0812 07:08:16.844189 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.844193 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.844263 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.844340 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:16.844800 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.845498 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.846824 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:16.847237 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:16.847347 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:16.847754 (+ 407us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:16.847857 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:16.847932 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:16.848109 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 07:08:16.848191 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260812 07:08:16.855863 21221 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:08:16.853658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56916 (local address 127.11.192.252:40457)
0812 07:08:16.853866 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:16.853873 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:16.854360 (+ 487us) server_negotiation.cc:407] Connection header received
0812 07:08:16.854540 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:16.854546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:16.854622 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:16.854707 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:08:16.855602 (+ 895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56916: BlockingRecv error: recv got EOF from 127.0.0.1:56916 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:08:16.885239 21177 catalog_manager.cc:1259] Loaded cluster ID: 8b3a2eab37044a2a878bf42ca3703235
I20260812 07:08:16.885327 21177 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4579b9b85afb4c178fd4ebab8ac4bccd: loading cluster ID for follower catalog manager: success
I20260812 07:08:16.887818 21177 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4579b9b85afb4c178fd4ebab8ac4bccd: acquiring CA information for follower catalog manager: success
I20260812 07:08:16.889567 21177 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4579b9b85afb4c178fd4ebab8ac4bccd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 07:08:17.120556 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.115610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37048 (local address 127.11.192.253:38101)
0812 07:08:17.115850 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:17.115856 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:17.115998 (+ 142us) server_negotiation.cc:407] Connection header received
0812 07:08:17.116258 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:17.116262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:17.116333 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:17.116413 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:17.116848 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.117586 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.118913 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.119331 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.119441 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.119853 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:17.119959 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:17.120038 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:17.120211 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 07:08:17.120293 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 07:08:17.130950 21221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.125928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56922 (local address 127.11.192.252:40457)
0812 07:08:17.126101 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:17.126106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:17.126338 (+ 232us) server_negotiation.cc:407] Connection header received
0812 07:08:17.126467 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:17.126471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:17.126624 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:17.126720 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:17.127087 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.127781 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.129035 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.129468 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.129562 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.130021 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:17.130529 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:17.130642 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:17.130680 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 07:08:17.130749 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:08:17.143561 21221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.137547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56930 (local address 127.11.192.252:40457)
0812 07:08:17.137694 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:17.137699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:17.138104 (+ 405us) server_negotiation.cc:407] Connection header received
0812 07:08:17.138356 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:17.138363 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:17.138440 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:17.138538 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:17.139139 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.139986 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.141679 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.142338 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.142460 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.142847 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:17.142964 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:17.143041 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:17.143187 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 07:08:17.143263 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 07:08:17.145267 21480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.136330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51570 (local address 127.11.192.254:39925)
0812 07:08:17.136939 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:17.136945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:17.137554 (+ 609us) server_negotiation.cc:407] Connection header received
0812 07:08:17.137694 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:17.137700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:17.137919 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:17.138072 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:17.138795 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.139958 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.142550 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.143193 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.143302 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.143957 (+ 655us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:17.144468 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:17.144567 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:17.144697 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 07:08:17.144782 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":355,"threads_started":1}
I20260812 07:08:17.145385 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:08:17.136364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37052 (local address 127.11.192.253:38101)
0812 07:08:17.136558 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 07:08:17.136564 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:08:17.137450 (+ 886us) server_negotiation.cc:407] Connection header received
0812 07:08:17.137871 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:08:17.137877 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:08:17.137961 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:08:17.138036 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:08:17.138880 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.139958 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.143300 (+ 3342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:08:17.143889 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:08:17.144005 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:08:17.144631 (+ 626us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:08:17.144763 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:08:17.144826 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 07:08:17.145030 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 07:08:17.145112 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:08:17.150071 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 20763
I20260812 07:08:17.150997 20871 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:08:17.258626 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 20763
W20260812 07:08:17.287837 21058 connection.cc:535] server connection from 127.0.0.1:56784 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 07:08:17.288144 20922 connection.cc:535] server connection from 127.0.0.1:36922 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 07:08:17.288496 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 20893
I20260812 07:08:17.289537 21000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:08:17.393512 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 20893
I20260812 07:08:17.420615 12035 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu pid 21020
I20260812 07:08:17.421725 21142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:08:17.524789 12035 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpv3LkS/build/asan/bin/kudu with pid 21020
2026-08-12T07:08:17Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4759 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9925 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (360806 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST